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Abstract. We consider convex maps / : R" that are monotone (i.e., 

that preserve the product ordering of R"), and nonexpansive for the sup-norm. 
This includes convex monotone maps that are additively homogeneous (i.e., 
that commute with the addition of constants). We show that the fixed point 
set of /, when it is non-empty, is isomorphic to a convex inf-subsemilattico of 
R", whose dimension is at most equal to the number of strongly connected 
components of a critical graph defined from the tangent affine maps of /. 
This yields in particular an uniqueness result for the bias vector of ergodic 
control problems. This generalizes results obtained previously by Lanery, Ro- 
manovsky, and Schweitzer and Federgrucn, for ergodic control problems with 
finite state and action spaces, which correspond to the special case of piece- 
wise affine maps /. We also show that the length of periodic orbits of / is 
bounded by the cyclicity of its critical graph, which implies that the possible 
orbit lengths of / are exactly the orders of elements of the symmetric group 
on n letters. 
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1. Introduction 



1.1. Motivations and Statement of the Main Result. We say that a map 
/ : R" K" is monotone if for all x,y G M", x < y f{x) < f{y), where < 

denotes the product ordering of R" (x < j/ if Xi < yi, for all 1 < i < n). We say 
that / is additively homogeneous if for all A € M, a; S M", /(A + x) = A + f{x), 
where A + a; = (A + xi,... ,A + a;„). It is easy to see that a monotone homogeneous 
map is nonexpansive for the su p-norm : for all x,y G R", — f{y)\ < l^: — y\, 

where |a;| = maxi<j<„ \xi\ (see |CT80 |). 

Monotone homogeneous maps arise classically in optimal control and game the- 
ory (see for instance |lBK76t |Koh80| |Whi86|, |Kol92i |FV97|, |RS01|), in the model- 



ing of discrete events systems (see 



BCOQ92 



Gun94, CGQ95a, CGQ95b, Vm97 



GG9S, |Gun01 |), and in nonlinear potential theory [Dcl9£], as nonlinear extension 



of Markov transitions. They also arise in nonlinear Perron- Frobenius theory, when 
one considers multiplicativcly homogeneous maps F acting on a cone and preserv- 
ing the order of the cone: in the simplest case, when the cone is (M*)" (where R* = 
{x e M I X > 0}), the transformation F t-^ logoF o exp (where log : (R*)" R" 

is the map which does log entrywise, and exp = log~^) sends the set of monotone 
multiplicatively homogeneous maps to the set of monotone additively homogeneous 
maps. See for instance [ lBou94 |LW94 |Mor64 |MS69| , |Nus88| , |Nus8g| , |Sab97| , |Woj85| 
for various studies and applications. 

A basic problem, for a monotone homogeneous map /, is the existence, and 
uniqueness (up to an additive constant), of the additive eigenvectors of /, which 
are the v G R" such that /(w) = X + v, for some additive eigenvalue A € R. In the 
sequel, we will omit the term "additive" , when the additive nature of the objects 
will be clear from the context. When / has an eigenvector v with eigenvalue A, by 
homogeneity of /, f^{v) ^ k\ + v holds for all fc > 0, and by nonexpansiveness of 
/, \f{x) -kX-v\ = \ f''(x) - ff'Ml <\x-v\, hence, 



(1) 



fix) = kX + 0{l) 



when fc — > oo, 



for all X S R" (all the orbits of / have a linear growth rate of A). This implies in 
particular that the eigenvalue A is unique. Hence, we can speak without ambiguity 
of the eigenspace of /, which is the set S'{f) = {x € R" | f{x) = A 4- x}. In 
many applications, the eigenvalue and eigenvector are fundamental objects: for in- 
stance, in stochastic control, the eigenvalue gives the optimal reward per time unit, 
and eigenvectors give stationary rewards (we explain this in detail in Section 
In discrete event systems applications, the eigenvalue gives the throughput, and 
eigenvectors give stationary schedules. 

Several Perron-Frobenius like theorems guarantee the existence of eigenvectors 
of monotone (additively) homogeneous maps R" — > R". Such results go back at 
least to Krem and Rutman [KR50, §7], in the context of monotone multiplica- 
tively homogeneous maps leaving a cone in a Banach space invariant, and to Mor- 
ishima, whose book |Mor64| contains a complete study of finite dimensional non- 
linear Perron-Frobenius theory. A modern overview appears in the memoirs of 
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Nussbauni [Nus88, Nus89||, wh ich contain in particular general existence results for 
eigenvectors, following |Nus86|. Different existence conditions appeared in [ psh84 |. 



General results on the geometry of the eigenspaces are available, for instance, the 
result of Bruck |Bru73 | shows in particular that <#"(/) is the image of a nonexpansive 

' ' Theorems 4.5, 4.6 and 4.7]. 



projector and a fortiori is connected, see also iMusg 

In this paper, we describe the eigenspaces of convex monotone homogeneous 
maps / : R" M". (We say that a R"-valued map is convex when its coordinates 
are convex. We refer the reader to |Roc70| for all convexity notions used in the 
paper: subdifferentials, domain, Fenchel transform, etc.) 

To state our main result, we need a few definitions (see Section ^ for details). 
We first generalize the notion of subdifferential to maps R" — > R" by setting, for 
X € R", df{x) = {P G M"^" I f{y) - fix) > P{y - x), Vy € R"}. It is easy to 
see (Corollary |2?^ and Equation (^) below) that by monotonicity and homogeneity 
of /, the elements of df{x) are stochastic matrices. If v is an eigenvector of /, 
we call critical graph of /, the (directed) graph ^^^^(7) which is the union of final 
graphs of stochastic matrices P € df{v) (we call final gr aph of a stochastic matrix 
the restriction of its graph to the set of final classes, see § ^.2| and § |2.3|) . The graph 
^'^{f) is independent of the choice of the eigenvector v (Proposition |2.5| below). We 
call critical nodes of /, the nodes of ?^'^(/), and denote by N'^(/) the set of critical 
nodes. We call critical classes of / the sets of nodes Ci, . . . ,Cs of the strongly 
connected components oi'^^{f), ^i,. . . ,^s- We call cyclicity of ^^i, and denote by 
c(^^i), the gcd of the lengths of the circuits of ^^i, and we define the cyclicity of / 
by c(/) = lcm(c($#i), . . . ,c($#s)). We say that a monotone homogeneous map g : 
U C M" ^ y C R^ is a monotone homogeneous isomorphism if it has a monotone 
homogeneous inverse. The following theorem gathers results from Theorem 3.4 
Corollary Corollary 5.7, and Theorem 



below. 



Theorem 1.1 (Convex Spectral Theorem). Let f : R" R" denote a convex 
monotone homogeneous map that has an eigenvector. Denote by C — N'^(/) the 
set of critical nodes of f , c = c(/) the cyclicity of f, and A the unique eigenvalue 
off. Then, 



the restriction r : R" R'^ , a; i-^ {xi)iizc, is a monotone homogeneous iso- 
morphism from (o{f) to its image (o''^{f); 
S"^{f) is an inf-subsemilattice o/(R'^,<).' 

'^'^(f) is a convex set whose dimension is at most equal to the number of 
critical classes of f , and this bound is attained when f is piecewise affine; 
for all X E R", f'"'{x) — kcX has a limit when fc — > oo. 



In particular, when / has only one critical class, the eigenvector of / is unique 
(up to an additive constant). It also follows from the last assertion of the theorem 
that the set of li mit points of f'^{x) — kX when k oo, and x G R", is precisely 
^{f^). Theorem 1.1 also allo ws us to bound t he dimension of this set. Indeed, 



we shall see in Theorem 4.1 and Proposition 5.3 below that the set of critical 
nodes is the same for / and f^, and that has c(^#l) + • • • + c(5^s) critical classes. 
Hence, applying Theorem 1.1 to f^, we get that the restriction r is a monotone 
homogeneous isomorphism from iS{f'^) to a convex set, S'^{f'^)^ of dimension at 
most c(^^i) + ■ ■ ■ + c($#5), the bound being attained when / is piecewise affine. 

The paper is devoted to the proof (Sections 2-6) and to the stochastic control 
interpretation (Section 7) of Theorem |l . l| . In Section ||, we detail the definitions 
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and properties of subdifFerentials and critical graph of convex monotone homo- 
geneous maps. An important element of the proofs is the maximum principle for 
Markov chains (Lemma |2.9| ). In Section ||, we establish the first part of Theorem 
concerning the structure of the eigenspace: points 1, 2 and the first assertion in 
point 3. The main argument is again the maximum principle. Section ^ is devoted 
to further tools and properties used in the remaining sections, which are of inde- 
pendent interest: Theorem 4.1 shows that '^^{f^) = '^'^{f)'^ (this will be used in 
Section ^ for the proof of the cyclicity part of Theorem |L1|; we also introduce direc- 
tional derivatives (which will be used in Section ^to connect <?(/) to ^(/^) for any 
eigenvector u), additive recession functions (formula (p^), invariant critical classes 
and the associated decomposition of / (Lemma ^.91 ), and also a characterization 
of the set of critical nodes in terms of supports of nonlinear "excessive" measures 
(Proposition |45| ) . 

Section ^ is devoted to the proof of poi nt 4 of Theorem 



1.1 



This result relies 
which states 



on a more general theorem of Nussbaum |Nus90| and Sine [ 3in90 
that if / ; R" is nonexpansive for the sup- norm and has a fixed point, 

then, for all x G K", f^'^{x) converges when fc — > oo, for some minimal constant 
c which can be bounded by a function of n. When / is convex monotone and 
homogeneous, the last assertion of the convex spectral theorem shows that the 
possible values of c are exactly the orders of elements of the symmetric group 
on n letters. Equivalently, convex monotone homogeneous maps have the same 
orbit lengths as permutation matrices, a result which was known to be true in the 
special cases of linear maps associated to nonnegative matrices (see [ NVL99 , chapter 
9]), of linear maps over the max-plus semiring (see |CDQV83| , [Nus91a[ ), and also 
of piecewise afRne convex monotone homogeneous maps (which include max-plus 
linear maps), see the discussion in §1.2 below. More generally, computing the orbit 
lengths of nonexpansive maps for polyhedral norms raises interesting combinatorial 
and analytical problems (see in particular |AK87, Wel87, SchSS, ^ch91 
Nus91b| , |NSVL98| , |NVL99| , |LemOl| ) 



Nus90 



The equality in point 3 of Theorem 1.1 is proved in Section As will be discussed 
2 , this part of the theorem has already been proved by Romanovsky | Rom73| 



and by Schweitzer and Federgruen |SF78|. We provide here an independent proof, 
which emphasizes the qualitative properties of (?(/), using the tools of Section ^ 
We also give a polynomial time algorithm to compute ^^'^(/). 



1.2. Related Optimal Control Results. Convex monotone homogeneous maps 
/ : M" R" are exactly dynamic programming operators associated to stochastic 
control problems with state space {1, . . . ,n}. Computing the stationary solutions 
and the asymptotic behavior of solutions of dynamic programming equations is an 
old problem of stochastic control which is essentially equivalent to that of com- 
puting the eigenspace and the asymptotics of J*' when k oo. This has 
been much studied in the stochastic control literature, particularly in the case of 
finite action spaces, which corresponds to piecewise afHne maps. In this special 
case, results equivalent to the third assertion of Theorem 1.1 were obtained by Ro- 



manovsky | Elom75|] usi ng linear programming techniques, and also by Schweitzer 



and Federgruen | SF7§| | who gave an explicit representation of (?(/) in terms of 



resolvents associated to optimal strategies (sec [SF7S, Theorem 4.1]). Again in 



this special result equivalent to the fourth assertion of Theorem was 



stated by Lanery [ lLan67[ . The arguments of |Lan67| only proved the special case 



SPECTRAL THEOREM FOR CONVEX MONOTONE HOMOGENEOUS MAPS 



5 



where N'^(/) = {l,--- ,n}, sec the discussion in the introduction and in Note 1 
of pF77[ . A pro of vaU d for a general N'=(/) C {1, . . . , n} was given by Schweitzer 
and Federgruen [ ^F77 |, who also proved the optimality of c(/). 

The special case of deterministic control problems leads to maps / that are 
max-plus linear. These maps have been studied independently by the max-plus 
community. In this context, the dimension of the eigenspace was characterized 
by Gondran and Minoux |GM77], and the remaining part of the max-plus spec- 
tral theorem, dealing with cyclic! ty, was obtained by Cohen, Dubois, Quadrat, 
and Viot [CDQV83 (see also [BCOQ92 ). (Note however that more precise re- 
sults -explicit form of the eigenspace, finite time convergence of the iterates- are 
available in the max-plus case.) The max-plus spectral theorem has a long story, 
which goes back to Cuninghame- Green (see |CG79] and the references therein), Ro- 
manovsky |Rom67|, and Voro byev |Vor67) , to quote the most a ncient contributions. 
See the collection of articles | MS92 1 , Kolokoltsov and Maslov | KM97[], and the rcf- 
erences therein, for generalizations to infinite dimension. Sec also [ GP97 , Bap98| 
for surveys. 

The present work was inspired by the max-plus spectral theorem and uses non- 



expansive maps techniques (we were unaware of the results of [Lan67, Rom73 
3F77, SF7^ ]). We next emphasize differences with earlier results. We consider gen- 
eral convex monotone homogeneous maps, which correspond to stochastic control 
pro blems with fini t e stat e spac e and arbitrary action spaces, whereas the results 
in | Lan67 , Elom75 , |SF77 , SF78| require the action space to be finite. Our proof 
technique, which relies on the maximum principle, can be naturally transposed 
to other (infinite dimensional) contexts. Another tool in our proof is the critical 
graph ^'^(Z), which generalizes the critical graph that appears in max-plus algebra 
(see Proposition pT7| ). The critical graph already appeared in | Elom73| , p. 491], 
with a different definition in terms of optimal policies. The new definition that we 
give here in terms of subdifferentials leads in particular to a polynomial time algo- 
rithm to compute ^#'^(/) (see §3.3). (The equivalence of both definitions is shown 
in Proposition [7.2[ ) It should also be noted that when passing from the case of 
finite action spaces to arbitrary action spaces, new phenomena occur. For instance. 
Example 3.9 shows that we cannot hope, in general, to characterize the dimension 
of eigenspaces in terms of graphs like '^^'^{f). 

Let us mention in passing that the critical graph has an intuitive interpretation 
in terms of "recurrence" . For a Markov chain, a node is recurrent if the probability 
of return to this node is equal to one. For a max-plus matrix with eigenvalue 0, a 
node is "recurrent" , i.e. belong to a critical class, if we can return to this node with 
zero reward. When / is a convex monotone homogeneous map with eigenvalue 0, 
a node i is "recurrent" , i.e. belong to a critical class, if we can find a strategy for 
which, starting from i, we eventually return to i with probability 1 and zero mean 
reward. This provides a new illustration of the analogy between probability and 
optimi zation develope d in |Mas73|, C hap. VIII], [|MTRS90| , |Qua90i |AQV95l |Mor95i , 
QP97| , §4.2], |.MSOO| , and [|PuhOf| . 

Ergodic control problems of diffusion processes lead to spectral problems for 
infinite dimensional monotone homogeneous semigroups which can be expressed 
in terms of ergodic Hamilton- Jacobi-Bcllman (HJB) partial differential equations. 
In |Bcn8^], Bensoussan proved uniqueness of the eigenvector (as weak solution of 
the ergodic HJB equation) under assumptions, which translated in finite dimension 
imply irreducibility of stochastic matrices P E df{v). Inspired by the results of 
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the present paper, the first author, Sulem and Taksar |AST01] proved uniqueness 
of the viscosity solution of a special ergodic HJB equation. This yields an example 
of concrete situation where some non optimal stationary strategies have several 
final classes, whereas the optimal ones have only one final class (translated to our 
setting, this means that for some x G M" and P € df{x), P may have several final 
classes, whereas there exists an eigenvector v such that all elements of df{v) have 
one final class). 

1.3. From Uniqueness Results to Existence Results. The uniqueness result 



that follows from Theo rem 1.1 can be thought of as a partial extension of the 
condition of Nussbaum |Nus8S, Theorem 2.5]: specialized to convex monotone ho- 
mogeneous maps / : M" M", the result of Nussbaum shows that if / is and 
for all eigenvectors u, f'{v) has only one final class (in this case of course / has 
a unique critical class), then the eigenvector of / is unique. The idea of all these 
results is that the dimension of «?(/) can be bounded by looking at "linearizations" 
of / near an eigenvector. 

It is instructive to note that the uniqueness of eigenvectors in M" is governed 
by the same kind of graph properties as the existence of eigenvectors, albeit the 
graphs are different. For instance, a result of the second author and Gunawar- 



dena | ]GG01 , Theorem 2] guarantees the existence of an eigenvector for a monotone 



homogeneous map which has a strongly connected graph. Here, the graph ^#(/) of 
a monotone homogeneous map / : M" E" has nodes {1, . . . , n} and an arc i ^ j 
if limy_,oo fiiyej) = +00, where Cj denotes the j'-th vector of the canonical basis of 
R". Another way to guarantee the existence of an eigenvector is to use the convex 



spectral theorem itself, thanks to the following observation taken from |GG01|. We 
denote by f{x) = lim^_,oo f(fix) the recession function of / (/ need not exist 
when / is monotone and homogeneous, but it does exist when / is convex). We 
have /(O) = 0, and when / is (additively) homogeneous, so does /, so that all 



points on the diagonal are (trivial) fixed points of /. It is proved in GG01 | that if 
the recession function of a monotone homogeneous map / exists and has only fixed 
points on the diagonal, then, / has an eigenvector. Combining this observation 
with the convex spectral theorem, we obtain: 

Corollary 1.2. A monotone homogeneous map has an eigenvector if its recession 
function exists, is convex, and has only one critical class. 

If / is a convex monotone homogeneous map, it is not difficult to see that the 
recession function / is exactly the support function of the domain of the Fenchel 
transform /* of / (defined in Section ^.l| below), that is f{x) = supp^^^^^, Px. 
In this formula, one can replace dom/* by its closure cl(dom/*), which is equal to 
9/(0). The graph ^^(/) is the union of the graphs of P e dom/*, or equivalcntly 
the union of the graphs of P G cl(dom/*), whereas the critical graph ^#'^(/) is the 
union of the final graphs of P € cl(dom/*). If ^(/) is strongly connected, one 
can see that 'i^^{f ) is also strongly connected, so that in the special case of convex 
monotone homogeneous maps, Corollary L2 is stronger than Theorem 2 of | GGOl| 



(which however holds in a more general context) 

1.4. Extension to Convex Monotone Subhomogeneous Maps. Finally, let 
us mention two immediate extensions of the convex spectral theorem. First, since 
the map f ^ [x ^ — /(— x)) sends convex monotone homogeneous maps to concave 
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monotone homogeneous maps, there is of course a dual concave spectral theorem. 
Another, more interesting, extension, is obtained by considering subhomogeneous 
maps /, which satisfy /(A + a;) < A + /(cc), for all A > and x G K". It is easy 
to see that a monotone map is subhomogeneous if, and only if, it is nonexpansive 
for the sup- norm. To a monotone subhomogeneous map / : E" M", we associate 
canonically a monotone homogeneous map g : R"+^ M"+^, 

(this is a nonlinear extension of the classical way of passing from a substochastic 
to a stochastic matrix, by adding a cemetery state, in this non-linear context, this 



construction is due to Gunawardena and Keane |GK95|). A vector z £ R" is a 
fixed point of /, if, and only if (z, 0) is an eigenvector of g (and the eigenvalue is 
0). Using this construction, one translates readily the Convex Spectral Theorem 
to a theorem describing fixed point sets and the asymptotics of the iterates of 
convex monotone subhomogeneous maps. (We might also use this construction, 
with —A -I- / instead of /, to describe the eigenspace of / for an additive eigenvalue 
A, but when / is only monotone and subhomogeneous, A need not be unique, and 
it tells little about the asymptotics of , in general.) For a convex monotone 
subhomogeneous map / with fixed point w, the critical graph ^^'^(/) of / is defined 
as the union of the graphs of the matrices PcCi where P G df{v), C is a final class 
of P, and the C x C submatrix of P, Pec, is stochastic (when / is homogeneous, 
this property is automatically satisfied). Equivalently, '^'^{f) (which can be empty) 
is the restriction of $#'^((7) to {1, . . . , n}. The notions of critical classes and cyclicity 
are defined from ?#'^(/) as above. When $#'^(/) is empty, we have N'^(/) ~ 0, and 
we take the convention M*^ = {0}, and c(/) ~ 1. 

Corollary 1.3. Let f : R" — > R" denote a convex monotone subhomogeneous map 
that has a fixed point. Then, all the conclusions of the Convex Spectral Theorem 
apply to f and A = 0. In particular, if f has no critical classes, then its fixed point 
is unique. 

Acknowledgements. The authors thank J. Gunawardena, J. P. Quadrat, and C. 
Sparrow, for many useful discussions. 

2. Class Structure of Convex Monotone Homogeneous Maps 

2.1. SubdifFerentials of Convex Monotone Homogeneous Maps. We shall 
first consider scalar monotone homogeneous maps g : R" R (which satisfy x < 
y =^ g{x) < g{y) for aU x,y & R", and g{\ + x) = X + g{x), for all A € M 
and X e R"). The Fenchel transform of g is the map g* : R" — > R U {-l-oo}, 
g*{p) = swpxiz-R^ijP ■ X - g{x)). We denote by domg* = {p E R" | g*{p) < 00} the 
domain of g* , and by = {p G R" | J2i<i<7iPi ~ 1; Pii • ■ • ^Vn > 0} the set of 
stochastic vectors. 

Proposition 2.1. If g : R" R is monotone and homogeneous, then, domg* is 
included in .S^n ■ 

Proof. Let 1„ denote the vector of R" whose entries are all equal to 1. li g 
is homogeneous, we have for all p G R", g*{p) > svip)^^^{p ■ Al„ — p(Al„)) = 
suPagr(-^Ei<»<„P* - a - g{0)) = sup^^^ \{J2i<^<nPi - 1) - 5(0)- Hence, 
g*{p) < 00 implies that J2i<i<nPi = 1- Similarly, if g is monotone, we have 
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g*(j') > sup^<q(p • X — g{x)) > sup3,<q(p • X' — g{0)). Hence, g*(ji) < oo implies that 
Pi,-- - ,Pn > 0. □ 

Corollary 2.2. If g : R" ^ R is monotone, homogeneous, and convex, then, for 
all V G M", g{v) ~ suppg(jo„-,g* {p - v ~ g*{p)), and the subdifferential 

9g{v) = {peM."\ g{x)-g{v)>p-{x-v),yxeR''} 
(2) = {p e domg* \ p- V - g*{p) ^ g{v)} 

is a non-empty compact convex subset of ,5^n- 

Proof. Since g is convex and takes only finite values, the subdifferential dg{v) is non- 
empty for any v £ R", and a subdifferential is trivially closed and convex. Moreover, 



dg(v) C domg* by definition of dg{v), hence, by Proposition 2.1, dg{v) C 



Finally, dg(v) is compact as a closed subset of the compact set S^n- CH 

Let S^nn denote the set oi n x n (row-)stochastic matrices. For any n x n matrix 
P, we denote by Pi, . . . P„ the rows of P and identify P to (Pi, ... , P„), which 
amounts to identifying R"^" to (R^^")" and ^„ to (^)". If / : M" ^ R" is 
convex, we set, for all v £ R", 

(3) dfiv) =^ {P G R"^" I fix) - f{v) > P{x - v), \fx e R"} . 

If / = (/i,... ,/„), the Fenchel transform of / is the map f* : P e R"^" ^ 
(/i*(Pi), . . . , f*{Pn)) e (RU {+oo})", and its domain is given by 

dom/* {P e R"""" I /*(P) e R"} = dom/* x • • • x dom/* . 

Of course 

(4) dfiv) ^dfi{v)x---x dfniv) C dom/* C . 
when / is monotone and homogeneous. 

2.2. Convex Rectangular Sets of Stochastic Matrices. We say that a subset 
^ of J^nn is rectangular ii ^ — !^\ x • • • x 1^^-, for some subsets !^\, . . . , ,f3^„ of 
5^ri- If / : — ^ R" is a convex monotone homogeneous map, the domain of /* 
is rectangular, and the subdifferential df(v) of / at any v S R" is rectangular. In 
this section, we extend the classical definition of graphs of stochastic matrices to 
the case of convex rectangular sets of stochastic matrices. 

Let us first recall some classical notions. For all graphs and nodes i,j of , 
we say that i has access to j , and we write i ~^ j, if there is a directed path from i 
to j in ^ , or if i = j. We call classes of the equivalence classes for the equivalence 
relation "(i ^ j) and (j i)". We call strongly connected component of ^, the 
restriction J^jc- of to a class C of that is the subgraph of J# with set of nodes 
C and arcs i ^ j when (i — > j) S and i,j € C. A graph is strongly connected if 
it has only one class. A class C of is final if no nodes of C have access to a node 
of the complement of C. 

To any nx n nonnegative matrix P, we associate the (directed) graph ^^(P) with 
nodes 1, . . . ,n and arcs i j when Pij 7^ 0. A matrix is irreducible if ^^(P) is 
strongly connected. Associating $#(P) to P allows us to use the graph vocabulary 
for nonnegative matrices, for instance the classes of P are by definition the classes 
of ^^(P). If P is a stochastic matrix, a class C of P is final if, and only if, the 
C X C submatrix of P is stochastic. For any stochastic matrix P, we denote N^{P) 
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the union of final classes of P, '^^^{P) the set of final classes of P and '^^{P) the 
subgraph of ^^(P) composed of the nodes and arcs of final classes of P, that is: 

(5) ^\P) = U ^(Pff) . 

F final class of P 

Here and below, for all n x n matrices P and subsets / and J of {1, . . . ,n}, we 
denote by Pij the I x J submatrix of P. If ^ is a rectangular set of stochastic 
matrices, we set 

Nf(^^)= y Nf(p), ■^f(^)= y 'rf^iP), ^f(^)= y ^f(p). 

Pg^ Pg^ Pg^' 

Hence, '^^^{^) has N^(,?3^) as set of nodes, and i j is an arc if there exists P G ^ 
such that Pij 7^ and i and j belong to the same final class of P. We say that a 
class C G ^^^{P) is maximal if it is maximal for inclusion. We have: 

Proposition 2.3. Let be a convex rectangular set of stochastic matrices. 

1. // C, C G (^) and Cr\C' i^Q), then, C U C e 

2. The maximal elements o/'^^(^^) are disjoint; 

3. There is a matrix P £ 3^ such that ^^^(P) = ^^^(^3^). The final classes of P 
are precisely the classes of '^^{3^). 

4. The classes o/^^^(^) are exactly the maximal elements of '^^{3^). 

Proof. By assumption, 3^ = x • • • x where ^^i, . . . , are convex subsets 
of 

If C,C' G 'i\3^), then, we can find P,P' e ^ such that C and C are final 
classes of P and P', respectively. Consider the matrix P" defined by P/' = (P; + 
P'')/2 if i G C n C", i^" = P^ if i G C \ C, i^" = P^' if i G C" \ C, and P^' = P, 
or P/ for i G {1, . . . ,n} \ (C U C") (the choice has no importance). Since the 
are convex, P/' G for all i, and by rectangularity, P" G To prove the first 
assertion, it remains to check that CUC" is a final class of P". By construction, the 
(C U C") X (C U C") submatrix of P" has row-sum 1, hence, we have only to check 
that all the nodes of CU C" are mutually accessible in ^^(P"). By definition of P", 
any path of ^(P) (resp. ^(P')) that remains in C (resp. C") is a path of ^(P"). 
Picking any node j G C n C", we see that there is a path from any i G C UC to j, 
and a path from j to i, in ^#(P"), which shows the first assertion. 

The second assertion follows readily from the first. 

For the third point, using the method of the first part of the proof, we shall 
construct by convexification a matrix Q £ 3^ such that ^^{3^) = ^^^(Q). For each 
arc i — > j of ^^^(^), there is a matrix P*-' G 3' such that i and j belong to the 
same final class P'^ of P'-', and {P^^)ij ^ 0. For each k G N^(,^), we consider 
= {iP'^)k I (i j) G ^f(^), k G P*^}, and we build the stochastic matrix Q 
whose fc-th row is given by Qfc = \^k\~\T,P(^s^ P) for all /c G (we denote by 

|-| the cardinality of a set), and Qfc = any element of .f^fc if fc ^ N^(^). By convexity 
and rectangularity of Q G 3^, hence ^^^(Q) C $#^(^). By construction, each 
row of Q with index in has sum 1, and ^^(Q) contains ^^{3^). Moreover, 

for any arc (fc I) of ^(Q) starting at fc G N^{3^), there exists (i — > j) G ?#^(^) 
such that {P'^^)ki > and k G P'^, hence fc and I are in P'-*, which implies that 
(fc ^ /) G ^^^(,^3^). In particular there are no arcs in 5^(Q) going out from N^(^), 
and the restriction of ^^(Q) to is equal to '!^^{3^). All these remarks imply 

that (Q) contains (J^*), so ^f(Q) = ^f(^). 
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Finally, any clement F oi'^^{3^) (a fortiori, any maximal clement) is such that 
two nodes of F are connected by a directed path in ^^^(J^), so F is included in a 
class of '^^{3^). Conversely, let C be a class of and take the matrix Q € 3^ 

of the third point. Then C is a final class of Q, hence C G "^^(c^), so C is included 
in a maximal element of Since the classes of '^^{3^) are disjoint, we obtain 

the last point of the proposition. □ 

Example 2.4. Let co{X) denote the convex hull of a set X, and consider the convex 
rectangular set ^ = x ^2 x ^3 C ^3, with 

^1 = co{(l, 0, 0), (1/2, 1/2, 0), (1/2, 0, 1/2)} , 
^2 = co{(0,l,0),(2/3,l/3,0)} , 
^3 -{(0,0,1)} . 

Since the identity matrix / belongs to {1}, {2}, {3} all are elements of '^^( 
Moreover, since 

' 1/2 1/2 
2/3 1/3 I G ^ 
1 

{1, 2} is also an clement oi'i^^{^). It is easy to see that these are the only elements 
of 'r^^{^), so that its maximal elements for inclusion arc {1,2} and {3}. The final 
graph 5^^(,?3^) of ^ is the following 

Di:^^20 03 

and its classes are {1,2} and {3}. We shall give, at the end of Section ^, an 
algorithm to determine the final graph of a convex rectangular sets with a finite 
number of extremal points. 

2.3. Critical Graph of Convex Monotone Homogeneous Maps. In the se- 
quel, / denotes a convex monotone homogeneous map that has an eigenvector. The 
associated eigenvalue will be denoted by A. As pointed out in the introduction, we 
may assume A = 0, so that statements and proofs will often be given in this case. 
We shall prove: 

Proposition 2.5. The sets N^{df{v)), '^^{df{v)) and the graph '^^{df{v)) all are 
independent of the choice of the eigenvector v of f . 

Thus, we shall simply write N'^(/), '^'^{f) and ^'^(/). We call critical nodes of / 
the elements of N'^(/), critical graph of / the graph ^^'^(/), and critical classes of / 



the classes of ?#'^(/). Combining Proposition 2.5 and Proposition 2.3, we get 



Corollary 2.6. 1. For any eigenvector v of f , there is a matrix P € df{v) 
such that '^'^{f) ~ '^^{P). In particular, the final classes of P are precisely 
the critical classes of f. 
2. The critical classes of f are exactly the maximal elements of'^'^{f). 

The graph ^'^{f) generalizes the classical critical graph that appears in max- 
plus algebra. Let us recall that if A is a n x n matrix with entries in M U {—00}, 
the critical graph 'i^'^{A) is the union of the circuits (ii,... ,ik) that realize the 
maximum in the formula: 



p{A) = max max 

l<k<n ii,... ,ifc 
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If A has at least one finite entry per row. the max-plus hnear map /a witlr matrix 
A, {fA{x))i = maxi<j<„(^y + Xj), sends R" into R". Of course, /a is monotone, 
homogeneous and convex. If w G M" is a max-plus eigenvector of A, with associated 
eigenvalue A (i.e. if = X + v), wc define the saturation graph of A and u, 

Sat(yl, v), as the union of arcs i ^ j such that X + Vi ^ Aij + Vj. 

Proposition 2.7. If A G (EU{— oo})"'^" /las least one finite entry per row, and 
if V £ M" is an eigenvector of A, then '^'^{fA) ~ '^'^{A) coincides with the union of 
strongly connected components ofSat{A,v). 

Proof. The fact that ^^'^ {A) coincides with the union of strongly connected compo- 



nents of Sat(yl, v) is a consequence of the max-plus spectral theorem (see [|BCOQ92 
§3.2.4 and 3.7]). The rest of the assertion follows from the identity {dfA{v))i = 
co{5j I («,j) G Sat(A,w)}, where 5j G ,5^n denotes the Dirac probability measure at 
j. Indeed, from this, one deduces that, for any P € dfA{v), ^{P) C Sa.t{A,v), 
so that ^^(P) is included in the union of strongly connected components of 
Sat(A, w), that is ^''{A). Hence, ^^Ua) C ^''{A). Conversely, for any circuit 
c in 'i§'^(^A) C Sat(A, one can construct P G dfA{v) such that = 1 for any 
arc i j oi c. The circuit c , which is a final class of P, is included in ^^'^(/a). 
Thus, ^%A) C ^'=(/a)- □ 



(We shall extend this property to the stochastic control context in Proposition 7.2.) 



Proposition 2.£ relics on classical facts of Perron- Frobcnius theory. We first need 
the characterization of supports of invariant measures. Recall that an invariant 
measure of a stochastic matrix P is a stochastic (row) vector m such that mP ~ m. 
The set of non-zero entries of m is the support of m. The following result is standard: 

Proposition 2.8. Let P be a n x n stochastic matrix. A subset F of {1, ... , n} 

is a union of final classes of P if and only if P has an invariant measure m with 
support F . □ 



The main ingredient of the proof of Proposition 2.5 is a classical property of 
super-harmonic vectors (a vector v is super-harmonic for a stochastic matrix P if 
Pv < v), which is itself a consequence of the Perron- Frobcnius theorem. 

Lemma 2.9 (Discrete Maximum Principle). If P is anxn stochastic matrix, and 
if Pv < V, then, the restriction of v to any final class of P is a constant vector, and 
Pv = v holds on the union N^(P) of the final classes of P. Moreover, the minimum 
of v is attained on a final class. 



Proof. Let C denote a final class of P. It follows from Proposition 2.8 that there 
exists an invariant measure m of P with support C. We have, < m{v — Pv) = 
mv — mv = 0, and since = m{v — Pv) ~ X]jec"^*(^'» ~ ^ sum of 

nonnegative terms, we must have Vi = {Pv)i for all i G C. This means that the 
restriction of w to C is a right-eigenvector of the C x C submatrix of P. Since this 
submatrix is stochastic and irreducible, this eigenvector must be constant, which 
shows the first part of the lemma. 

To show the second part, we introduce F = N'(P) and T = {1, ... ,n} \ F. 
We denote by Q the T x T submatrix of P, and by w the restriction of v to T. 
Let fi denote the minimum of the entries of v restrained to P. Since Pl„ = 1„, 
t; — /i > P{v — fi), which allows us to assume that ^ = 0. Then, v > Pv and 
> on P yield w > Qw, and since Q has nonnegative entries, w > Qw > Q^w > 
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. . . > Q'^w, for all k > 1. Since Q has spectral radius strictly less than 1, we get 
w > linife Q'^w = 0. □ 



Wc finally prove Proposition |2^. Let v,v' denote two eigenvectors of /. Let 
C e ^^^{df{v)) and let P S df{v) for which C is a final class. We get v' — v — 



/(«') — /(«) > P{v' — v). It follows from Lemma 2.9 that {v' — v)i = Pi{v' — v) holds 
for aU i e C. But f,{v') - f,{v) = {v' - v)i = Pi{v' - v) and P, e dfi{v) imply that 
P, e dU{v') (indeed, - h{v') = h{x) - U{v) + h{v) - U{v') > P,{x - v) - 

Pi{v' — v) > Pi{x — v') holds for all x G K"). Now, pick any Q S df{v'), and consider 
the matrix R such that Ri = Pi for alH € C and i?^ = if i £ {1, . . . , n} \ C. By 
rectangularity of df{v'), R e df{v'). Since C is a final class of i?, C G "^^{dfiv')). 
This shows that '^^{df{v)) C '^f^{df{v')). Since i?cc and Pcc coincide, this also 
shows that C and thus N^{df{v)) C Nf(9/(w')). □ 

Example 2.10. Let / : ^ 

/ xi V ^(xi + X2) V (-3 + X3) V i(.Ti + ss) 
(6) fix)={ .T2Vi(2.Ti+X2)V(-l + a;i) 

V xgV (-2 + 2:1) 

which is such that /(O) = 0. Wc have 9/(0) = with ^ defined in Example 2.4 
(that is (9/j(0) = for i = 1,2,3). So, the critical graph '^"{f) of / is exactly 
the final graph of ^ shown in Example |2.4 and the critical classes of / are {1,2} 
and {3}. Let us illustrate the fact that 'i^^{df{v)) is independent of the choice of 
the eigenvector v by considering the eigenvector v = (0,0,-2). By comparison 
with 9/(0), only the first and third rows of df{v) are changed, namely dfi{v) = 
co{(l, 0,0), (1/2, 1/2,0)} and dh{v) = co{(0, 0, 1), (1, 0, 0)}. Although df{v) ^ 
9/(0), the final classes of the matrices P E df{v) arc the same as the ones of the 
matrices P G 9/(0) = ^. 

3. Structure of Eigenspaces 

In order to make more apparent the proof idea, wc first show a simple result. 

Theorem 3.1. The eigenvector of a convex monotone homogeneous map with a 
unique critical class is unique, up to an additive constant. 

Proof. Let C denote the critical class of /, and let v, v' be two eigenvectors of /. 
Using Assertion ^ of Corollary p.6| , we get matrices P G 9/(w) and P' G df{v') 
such that C is the unique final class of P a nd P' . Since P G df{v), v' ~ v ~ 



f{v') — f{v) > P{v' — v), hence, by Lemma 2J, v' — v is constant on C, and it 
attains its minimum in C. Exchanging the roles of v' and v, we see that v' ~ v 
attains its maximum in C . Therefore, v' — v is a, constant vector. □ 



Example 3.2. Let us use Theorem to show that the map f ■.M?' ^ M^, 

(\{xi+X2) \ 
log(e^2 +8e^-^) 
a;iV(-l + X3) j 

has a unique eigenvector, up to an additive constant. Since the graph ^#(/) of /, 
which is equal to 
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is strongly connected, we known, thanks to |GG01, Theorem 2], that / has an 
eigenvector. Indeed, f{v) ^ X + v, where A ~ log2, and v ~ (log 2, 3 log 2, 0). We 
have 

/ 1/2 1/2 
df{v) ^ {f'{v)} where f'{v) = [ 1/2 1/2 

\ 1 

Thus, the critical graph ^'^{f) ^^(fiv)) = ^(/'(w)) is 

(only the arc 3^3 vanished, by comparison with $#(/)). Since ^^'^(/) is stro ngly 
connected, / has a unique critical class (namely {1,2,3}), and, by Theorem 3.1 , 
the eigenvector of / is unique (up to an additive constant). Equivalently, the map 
F : (RXf (K*)^ F = expo/ o log. 



y/XlX2 
F{x) = I X2+ SX3 

xi V (xs/e) 

has a unique (multiplicative) eigenvector, up to a constant, namely, (2, 8, 1). 



To extend Theorem 3.1 to the case of several critical classes, we shall use the 
following fundamental observation, which establishes the existence of a monotone 
homogeneous projector (—A + f)'^ from the super- eigenspace of /, <S^[f) = {x £ 
I fix) < A + x}, to the eigenspace S{f) = {x G M" | J[x) = A + a;}. 

Lemma 3.3 (Spectral Projector). // / : M" M" is a convex monotone homoge- 
neous map with eigenvalue 0, and if z (z M" is such that f{z) < z, then 

r(z) 1^' lim f{z) 

k — voo 

is an eigenvector of f , which coincides with z on the union N''(/) of critical classes 
of f . The map : S'~^{f) <^(/) monotone, homogeneous, convex, onto, and 
satisfies [f^f = f^. 

Proof. By monotonicity of /, the sequence Zk defined by Zq = z, Zk = /(zfc_i) for 
/c > 1, is nonincreasing. Picking any eigenvector v of / and using the nonexpansive- 
ness of /, we get \zf; — v\ ~ \f''{zQ) — /*''(^)l — No ~ '^l- Therefore, the sequence 
which is bounded and nonincreasing converges to a limit /"(z), which by continuity 
of / is an eigenvector of /. To show that f^iz) coincides with z on C = N'^{f), we 
observe that, for all u e M", 

(7) f{u) <u =^ f{u) = uotlC . 



Indeed, arguing as in the proof of Theorem 3.1, we can find, by Corollary 2.6 



matrix P € df{v) which has C as union of final classes. Then, u — v > f{u) 



/(f) > P{u — v), and by Lemma 2.9, u — v = P{u — v) on C, which implies 
that u — V ~ f{u) — f{v) on C, and since v — f{v), we get (^. Applying (|^) 
to the inequalities /(zq) < zq, f{zi) < zi, ... , we get zi = /(zq) = zq on C, 
Z2 = f{zi) = zi on C, . . . , hence Zk ~ zq on C for all A: > 0, which implies that 
/"(z) = hmfe zfe = z on C. Finally, : S'+{f) S{f) is onto since /"(x) = x 
when X € ^{f), and /", which is a pointwise limit of the monotone, homogeneous, 
and convex maps /'^ is also monotone, homogeneous, and convex. □ 



14 



MARIANNE AKIAN AND STEPHANE GAUBERT 



The supcr-eigenspace S''^{f) will be useful in the subsequent proofs, because S^i^f) 
is a "more regular" object than <S{f). In particular, thanks to the monotonicity 
of /, (S^{f) is an inf-subsemilattice of (IR",<), and, when / is convex, S'^{f) is 
convex. If C is any subset of {1, . . . , n}, we denote by MS^ the set of vectors indexed 
by the elements of C and by rc the restriction map M" Mp , rc{x)i = Xi, for all 
i e C. 

Theorem 3.4. Let Ci, . . . , denote the critical classes of a convex monotone 
homogeneous map f, and letC = CiU. . .UCg = N'^(/). Then, any two eigenvectors 
v,v' of f satisfy: 

1. V ~ v' is constant on each Ci; 

2. if V — v' on C, then v = v' . 
Moreover, 

3. the restriction rc sends bijectively S'{f) to a convex inf-subsemilattice ofMp, 
denoted by (S"^{f), and the inverse map : (S''^{f) <^(/) is monotone and 
homogeneous. 



Proof. Let v, v' be two eigenvectors of /. Arguing as in the proof of Theorem 3.1 



we get two matrices P G df{v) and P' £ df{v') which both have exactly the final 



classes Ci, . . . , Cg. Since v' — v > P(v' — v), we get from Lemma 2.9 that v' — v 
is constant on each final class Q, which proves the first assertion. Moreover, if 
v' — 1) > on C, the vector v' — u, which, by Lemma 2.9, attains its minimum on a 
final class, must be nonnegative. By symmetry, if w — w' > on C, w — v' must be 
nonnegative. Thus, v = v' otiC implies v = v' , which shows the second assertion of 
the theorem. This assertion implies that rc is injective ^(/) <^"^(/)- The inverse 
map r^^ is trivially homogeneous. It is also monotone: if w' = rci^') > w = rc{v), 
v' — V > on C, and we just saw that this implies that v' — v > 0. Thus, 
w' >w =^ r^^{w') > r^\w). 

Take now v,v' two eigenvectors of /, and assume that the eigenvalue of / is 



0. By monotonicity of /, f{v A v') < v A v' , and by Lemma 3.3, /"(w A v') is an 
eigenvector which coincides with vAv' on C. This shows that iS"^{f), equipped with 
the ordering <, is an inf-semilattice, in which the greatest lower bound coincides 
with that of R"^. 

Finally, let < a < 1, and a' ~ 1 — a. We have, by convexity of /, f{av + a'v') < 
af{v) + a' f(v') ~ av + a'v' , and arguing as for v A v' , we derive from this that 
<S^{f) is convex. □ 



Of course. Theorem 3.4 implies that the eigenspace is connected {S{f) is 



the im age of the convex set S"^{f) by the continuous map r^^). More generally. 



Bruck I Bru75 1 proved that the set of fixed points of a nonexpansive map in a Banach 
space is a retract of the whole space, and, a fortiori, is connected. In the special case 
of convex monotone homogeneous maps, we have more, since Theorem shows 
that the eigenspace S'{f) is isomorphic (by monotone homogeneous bijections) to 
the convex set S'^'if). This makes the following definition natural. 

Definition 3.5. The dimension dmiS{f) of the eigenspace of a convex mono- 
tone homogeneous map / is the dimension (as a convex set) of its restriction S"^{f) 
to the union of critical classes. 

(Recall that the dimension of a convex set is defined as the dimension of its 
affine hull). A perhaps more convenient way to describe the geometry of (o{f) is 
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to introduce the notion of section, which is any set S C N'^(/) which meets each 
critical class at exactly one point. We denote by m(/) the number of critical classes 



of /. Then, paraphrasing Theorem 3.4, we get: 



Corollary 3.6. Let f denote a convex monotone homogeneous map f with section 
S. Then, S"^{f) and rs(<^(/)) both are convex sets of dimension dim(S'(/) < m(/). 

A natural question is to characterize the dimension of (?(/). We shall see that 
the equahty dim#(/) = rn(/) holds, except in "singular" cases. Let us first exhibit 
a regular case. 



Example 3.7. For the map / given in we saw in Example 2.1C that the critical 



classes of / are {1,2} and {3}. In particular, m(/) = 2. Since u = is an eigen- 



vector, the first statement of Theorem 3.4 shows that any eigenvector is constant 



on {1,2}. Then, an elementary computation shows that (§'(/) = {x e R'^ | f{x) 
cc} = {x e R3 I xi = a;2 > 0:3 > -2 + xi}. Thus dimS'if) = 2 = m(/). 



Example 3.8. It is instructive to apply Theorem 3.4 or Theorem p.l| to the case of 
the map f{x) = log Af exp(2;), i.e., fi{x) = log(^^ M^j exp(xj)), where M is an 
irreducible nonnegative matrix M. We have df{x) = {P{x)}, where P{x) = f'{x) 
is the n X n matrix with entries P{x)ij = M^/j exp(a;fe))~^Afij exp(a;j). Since 
M is irreducible, P{x) is irreducible, for all values of x. Therefore, / has a unique 



critical class, and Theorem 3.1 tells that its eigenvector is unique, up to an additive 
constant, or, equivalently, that the matrix M has a unique positive eigenvector, up 
to a multiplicative constant. This is a (complicated) way to derive the uniqueness 
part of Perron-Frobenius theorem for irreducible nonnegative matrices, from the 
Perron-Frobenius theorem for irreducible stochastic matrices. 

The next example illustrates the difference between regular and singular cases. 

Example 3.9. Let h denote any convex map R ^ M whose subgradients are between 
and 1. The map / : R^ ^ R^, 



(8) /(x) 



xi + h{x2 - Xi) 
X2 + h{xi - X2) 



is monotone, homogeneous, convex, and /(O) — h{0) + 0, which means that is 
an eigenvector of / for the eigenvalue h{0). If {h{t) = h{—t) =^ t ~ 0), then 
is the unique eigenvector of /, up to an additive constant. This condition is 
satisfied by the maps t 1-^ log(l + e*) and t 1^ V t. It is also satisfied by the 
Moreau-Yoshida regularization, he, of t V t, which, for all e > 0, is defined by 
he(t) = for t < 0, he{t) = t^/2e for < i < e, and he{t) = t ~ e/2 for t > e. 
Let g, k, and k'^ denote the maps / obtained by replacing h hy t log(l + e*), 
t V i, and /i, in (|), respectively. We have dgi{0) = {(ffi)'(O)} = {(1/2, 1/2)}, 
5fci(0) = co{(l,0),(0, 1)}, and(9/cf(0) = {(fc^)'(O)} = {(1,0)}. Using the symmetry 
between the first and the second coordinate of /, we get the following critical graphs 

^=(g) =^=(A:) = Cl "'^ ^20, '^^{k')= Cl 2O . 



Thus, dimS'ig) = m{g) = 1, dim(?(fc) = m(fc) = 1, but dim^f (fc^) = 1< m(fc") = 2. 
This discrepancy should be intuitively clear by looking at the following graphs: 
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regular regular singular 

In the case oi f = k'^, Theorem fails to characterize the dimension of be- 
cause the subdifferential of / does not give enough information on the local behavior 
of / near the eigenspace. In such cases, one needs to consider terms of higher order 
in the local expansion of / to establish the uniqueness of the eigenvector. However, 
we shall see in section || that the equality dim<o'(/) = rn(/) does hold when / is 
piecewise affine. 

The following last example shows that need not be convex or an inf- 

subsemilattice of R", even if its restriction S''^{f) is a convex inf-subsemilattice 
of R^. 



Example 3.10. Consider / : 



X2 V a;3 



X3 

We have (?(/) ~ {{x2 V 2:3, 0:2, 2:3) | 2:272^3 G R}, / has two critical classes Ci = 
{2}, C2 = {3}, so dim^(/) = m(/) = 2. Let u = (1,1,0), v = (1,0,1). Since 
u,v € S{f) but {u + v)/2 = (1,1/2,1/2) ^ #(/), S{f) is not convex, and since 
u^v^ (1, 0, 0) ^ S{f), S{f) is not an inf-subsemilattice of R'^. 

Finally, we note that fixed points sets of monotone noncxpansive maps R" — > R" 
are always lattices: the interesting lattice statement in assertion of Theorem 3.4 
is that the inf law of S^{f ) coincides with the usual inf law in R . 

Proposition 3.11. Let f : R" R" denote a monotone map that is nonexpansive 
(for any norm) and that has a fixed point. Then, the fixed point set S'{f) ~ {x €z 
R" I f{x) = x}, equipped with the standard order relation, is a lattice, in which the 
sup and inf laws, V/ and Af, respectively, are given by 

(9) x\/fy= lim f''{x V y), xAfy= lim f''{x A y) . 

k — ^00 k — 'OD 

Proof, li x,y E <S'{f), then by monotonicity of /, f{x Ay) < f{x) A f{y) ^ x Ay. 
Arguing as at the beginning of the proof of Lemma |3.3| , we get that the limit 
w = linifc^oo f^{x A y) exists and satisfies f{w) = w, w < x, and w < y. Moreover, 
if z is an arbitrary element in (?(/) such that z < x, and z < y, we have z = 
f{z) ^ ■ ■ ■ = f^{z) < f^{x A y) for all fc, so that z < w. This shows that w = 
limfc_+oo f^{xAy)~xAf y. The dual argument shows the first equality in (^). □ 

4. Critical Graph of 

In this section, we establish the following result which is central in the proof of 
the cyclicity theorem. If ^ is a graph, we call 'i^^ the graph with same nodes as 
and arcs (i —>■ j) when there is a directed path i ^ ii ■ ■ ■ ^ ik — j with length 
k in ^ . 

Theorem 4.1. Let f denote a convex monotone homogeneous map R" — > R" that 
has an eigenvector. Then, for all fc > 1, ^^'^(/*'') = '^'^{f)^ , in particular N'^(/'^) = 
N^(/)- 



SPECTRAL THEOREM FOR CONVEX MONOTONE HOMOGENEOUS MAPS 



17 



The proof of Theorem 4.1 needs tools and results, of independent interest, which 
involve one-sided directional derivatives, additive recession functions, and a non- 
linear generalization of invariant measures. Let us first show the linear version of 



Theorem 4.1 



Proposition 4.2. If P is a stochastic matrix, then, for all k > 1, = 

^f(P)'=, in particular W{P^) = H^{P). 

Proof. For all nonnegative matrices P, we have ^{P'^) = '^{P)'^ . Since '^^{P) is 
equal to ^(Q), where Q is the restriction of P to N^(P) and since the restriction 
of P^ to N^(P) is equal to (by definition of final classes), it is enough to prove 
that H^{P^) = W{P). 

To show this, we shall use the following consequence of Proposition ^.8[ : for any 
stochastic matrix P, the support of an invariant measure m of P is a subset of 
N^(P), and there exists an invariant measure m of P with support N^(P). 

Consider first an invariant measure m of P with support N^(P). The vector m is 
also an invariant measure of P^ , which implies that its support N^(P) is included in 
N^(P'^). Conversely, consider an invariant measure m of P''' with support N^(P'^). 

We get m{I + h P''~^)P = m(P H h P'') = m{I H h P''"^), so nik = 

■^rn{I + • • • + P'^^^) is an invariant measure of P. Hence, the support of irik is 
included in N^(P). Since irik > \m > 0, due to the nonnegativeness of P and to, 
the support of contains that of in, which leads to N^(P'^) C N^(P). □ 

For any convex map / : R" M. and any v £ M", we denote by the 
(one-sided) directional derivative of / at w with respect to y: 

I'M- lim^^(!i±M-M . 

The map is well-defined, finite, convex, multiplicativcly homogeneous, and 
(10) /^(y) = sup p-y , 

p&dfiv) 



see | Roc70 , Theorems 23.1 and 23.4]. When / is monotone and additively homo- 
geneous, so does f^. In the sequel, we shall say that a map is bihomogeneous if it 
is both additively and multiplicatively homogeneous. 

The definition and properties of can be extended to convex maps / : M" 
W^, with /,', = ((/i );,...,(/„);) : M" ^ M™. We have the following chain rules: 

Lemma 4.3. Let f : M" M™ and g : K™ be two convex maps. Assume 

that g is monotone. Then, g o f : R" — > is convex, and, for all v G R", 

(11) = 9'f{v)°fv , 

(12) digof){v) = co{dgifiv))dfiv)) . 

(Recall that co denotes the convex hull. In (|l|), dg{f{v))df{v) {PQ \ P G 
dg{f{v)), Q e dfiv)}.) 



Proof. The convexity of g o / is immediate, and Eqn ( p_lD follows easily from the 
fact that any finite convex function is locally Lipschitz continuous. 



To show (12), we first note that if £> is a rectangular subset of R™, i.e. if D 



is of the form Di x • • • x and if h : R™ W is monotone and satisfies 
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/i(limfc Xk) — linifc h{xk) for all nondccrcasing sequences Xk^ then, 



(13) 



/i(sup d) ~ sup h{d) 

deD deD 



Indeed, the inequality > in ( [Tsl ) follows from the monotonicity of h, and, thanks 
to the rectangularity of Z?, we can find a nondecreasing sequence £ D such that 
suPdeijC? = liiUfedfe, hence, h{s\ip^f^jj d) = h{limkdk) = liTakh{dk) < sup^^^, 
which shows (USh. 

Applying to D = df(v)y and h{x) = Px, with P G dg{f{v)), we get 
from (|ll|) and (pi). 



(5°/);(2/)-5^(.)°/^(2/) 



(14) 



sup P{ sup Qy) 

P&dg{f{v)) Qedfiv) 

sup Ry . 

R<^dg{f{v))af{v) 



Let us denote by ■ K" ^ M U {+00} the support function of a subset X C M", 
which is defined by S\-{p) = sup^gj^ p • x. We first assume that ^ = 1. Then, (14) 
and property ( p^ ) show that (g o /)', = S[ 
Fenchel duality, two subsets of 



° fYv = ^kgomv) = ^*dgifiv))afivy By Legcndre- 
IR" have the same support function, if, and only 



if, they have the same closed convex hull (see [Eloc70, Corollary 13.1.1]). Since 
d{g o f){v) is closed and convex, d{g o f){v) = co {dg{f {v))d f (v)) where co de- 
notes the closed convex hull of a set. But dg{f{v))df{v), which is the image by 
the continuous map (P, Q) PQ of the product of two compact sets, is com- 
pact, and the convex hull of a compact subset of R", which is compact, coin- 



cides with its closed convex hull (see for instance [ AB99|, C orollary 5.18]), therefore 
cd{dg{f{v))df{v)) = co idg{f{v))df{v)), which shows (|l^), when £ = 1. The proof 
for i > 1 follows readily from the result when £ = I, together with the observation 
that the convex hull of a rectangular set is rectangular. □ 



We get as an easy corollary one inclusion in Theorem 4.1 



Corollary 4.4. // / is as in Theorem 4.1, and if > 1, then, df''{v) = 
CO ((3/(u))'^), for aU eigenvectors v of /. Moreover, '^^{f^) D '^'^{f)'^- 

Proof. By (^, df^v) = co{d f (v) coO_co d f (v) ■ ■ ■ )) = co{d f (v) ■ ■ ■ d f (v)) = 
CO ((9/(w))'^). Assertion H of Corollary |2.6| shows that there is a matrix P G df{v) 
such that '^%f) = ^'(P). Since P'^ e (dfiv))'' C df'iv), <^^{P^) C '^"{f^), and, 
using Proposition HJ, we get '^''{ff = '^^^{Pf = ^^(P'^) C '^"{f''). □ 



To show the other inclusion, we shall need the following nonlinear version of Propo- 



sition 2.8, which relates final classes with invariant measures. 



Proposition 4.5. Let f denote a convex monotone homogeneous map M" M" 
that has a fixed point, v, and let F be a subset of {1, ... , n}. The following propo- 
sitions are equivalent: 

1. F is a union of elements of'^^'^{f), 

2. there exists a stochastic row vector m with support F , such that mf{x) > 

3. there exists a stochastic row vector m with support F, such that mf^{x) > 
mx, V.T e M". 
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Proof. Without loss of generality, we assume that v = (otherwise, replace / by 
-v + f{- + v)). 

(1 =^ 2) Let be a union of elements of '^"{f) = 'if^ {d f (0)) . Using the 
first assertion of Proposition |2.3| , we can write F as the union of disjoint sets 
Fi,... ,Fk e'^'^if). Moreover, there is a matrix P G 5/(0) such that Fi, . . . , Ffe 
are final classes of P. Indeed, pick p(^) , • • • , P^*^^ G dfi^) such that for alll < Z < fc, 
Fi is a final class of P^^\ We denote by P the matrix whose i-th row is equal to 
the i-th row of P'-^\ when i £ Fi, and to any element of dfi{0), when i ^ F. By 
rectangularity of 9/(0), P G 9/(0), and Fi, . . . ,Fk are final classes of P. 



Thus, F is a union of final classes of P. It follows, by Proposition |2.8| , that P 
has an invariant measure m with support F. Since P G 9/(0), we get f{x) > Px 
for all X G M", so mf{x) > niPx = nix. 

(2 =^ 3) Assume that there exists a stochastic row vector m with support F, 
such that mf{x) > mx for all x G M". Then, me^^ f{ex) > mx, for all e > 0. 
Taking the limit when e goes to zero, one gets mfl^{x) > mx for all x G M". 
(3 1) Assume now that there exists a stochastic row vector m with support F, 
such that m/g(a;) > mx for all x G K.". Applying Property (|l0| ) to the coordinates 
of /q and using the rectangularity of 9/(0), we get 

mf'^(x) ~ m sup Px ~ sup mPx . 

Pedfio) Pedfio) 

Hence, setting ^ = {mP, P G 9/(0)}, we see that the assumption on m is 
equivalent to sup^^^^ vx > mx for all x G M". In order to prove point 1, it is 
sufficient to show that there exists P G 9/(0) such that mP = m, that is to G 



Indeed, by Proposition 2.8, this would imply that F is a union of final classes of P, 
thus a union of elements of 'i"^(/). Assume, by contradiction, that to ^ Since 
9/(0) is a convex compact subset of R"^", ^ is a convex compact subset of the 
space R^^" of row vectors. Hence, the Hahn-Banach theorem implies that there 
exists a separating hyperplane, that is there exists a column vector x G K" and a 
real a, such that mx > a and vx < a for all v G Then, sup^g^ vx < a < mx, 
a contradiction. We have proved that m G which shows point 1. □ 

We shall also need the following notion of additive recession function. Let g : R" — > 
R be an additively subhomogeneous map, that is a map such that g{\^-x) < X+g{x) 
for all A > and x G R". The sequence —p + g{p + y) is nonincreasing with respect 
to p > 0, allowing us to define the additive recession function g : R" ^ R U {— oo} 
of g by 

(15) g{y)^ lim -p + g{p + y) ■ 

If g is monotone and convex, so does g. Moreover, the domain of g* is included in 
the set .5^" = {p G R" | X]i<i<nP« < 1: Pi: ■ • ■ :Pn > 0} of suhstochastic vectors, 
and g is given by: 

(16) g{y) = sup q-y- 9*{q) , 

g(Edom g* nj5^ 

where an empty supremum is equal to — oo. By (|l5|) and the nonexpansiveness of 
g, g is either finite everywhere or identically equal to -co. 

We define similarly the additive recession function of an additively subhomo- 
geneous map g : R" ^ R". If ^ = (51,... ,gn), we have g = [gi,..- ,g^)- 
If g is monotone and convex, and if g has only finite entries, we have, by (iq). 
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g{p + y) = p + g{y) + o(l). when p goes to +00. So using the noncxpansivencss of 
5, we get g o h{p + y) ~ g{p + h{y)) + o(l) = p + g o h{y) + o(l), for any other map 
h with the same properties as g, which leads to the foUowing chain rule: 

Lemma 4.6. Let g and h : M" — > M" be two monotone, convex and subhomo- 
geneous maps, such that g and h take finite values. Then, g o h : M" — > M" is 
monotone, convex, subhomogeneous, and g o h = g o h. 

For any n x n stochastic matrix P, we say that the set -F C {1, . . . , n} is invariant 
by P if Pij = for alH £ F and j ^ F (that is, if it is invariant by the dynamics 
of the Markov chain of transition matrix P). 

Lemma 4.7. Let f denote a convex monotone homogeneous map M" —>■ K." that 
has an eigenvector v. Then, there exists at least one critical class of f that is 
invariant by all the matrices P £ df{v). We say that such a critical class is 
invariant by f. 

Proof. Let = Upggj^(,u)^#(P) and F denotes an arbitrary final class of The set 
F is clearly invariant by all the matrices P S df{v). Since is finite, there exists a 
finite set 3^ C df{v) such that ^ = Upe<^^(P). Consider P = I^THEpe^^ P)- 
Using the convexity of df{v), we have P g df{v). Since the graph of P is equal to 
F is a final class of P. hence F G '^'^{f). This is a critical class of / since for all 
P e df{v), P^j ^ for allie F and j ^ F. □ 

Example 4.8. For the map / : ^ R-^, 

X2 V 

both {2} and {3} are critical classes, but only {3} is invariant. 



The last ingredient in the proof of Theorem |4.1| is a simple restriction operation. 
If iV is a subset of {1, . . . , n}, we denote by \n the trivial injection M", 
which is defined by {\N{x))i = Xi ii i G N, and {\N{x))i = 0, otherwise. We 
define the restriction f^N '■ ^ of / to by /tvat = tat o / o i^. If / 
is convex monotone and bihomogeneous, then /atat is clearly monotone, convex, 
multiplicativcly homogeneous, and additively subhomogeneous. Indeed, f{x) = 
.foix) = suppgg^(o) Fx, so fNwiy) = supQgg^^^(o) Qy, with 9/Arw(0) = {Fata, | 
F € 9/(0)}. Recall that the critical graph of an additively subhomogeneous map 
has been defined in § |l.4[ We have the following lemma, which follows readily from 
the fact that for any F E 9/(0), Pfn = 0, so that any final class of F is included 
either in or in F. 

Lemma 4.9. Let F denote an invariant critical class of a convex monotone biho- 
mogeneous map f : K" —^ R", and let N ~ {I, . . . ,n}\F. Then, /o Iat = \n° fNN, 
fpF ° ~ fp ° f o,nd the map fpp is additively homogeneous. Moreover, 
'^'^(f) ~ '^'^ifNN)^'^'^ifFp), and, when all the entries of f^jy are finite, 

(17) ^'if)^^'{hrN)U^'ifFF) 

(the unions are disjoint). □ 

In general, fNN is only additively subhomogeneous, since the N x N submatrices 
FjvAf are only substochastic. 
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Example 4.10. For the map / of Example 4.8, we have F = {3}, N = {1, 2}, and 



Inn 



Xl 
X2 



X2 
X2 V 



Inn 



Xl 
X2 



X2 
X2 



and $^'^(/) = 'i^'^ifNN) ^'^'^{fpF) is the disjoint miion of the two graphs consisting 
of a single loop at node 2, and 3, respectively. 



Proof of Theorem (.1 
of/ 



Without loss of generality, we assume that is a fixed point 
By definition of = ^'^(/o). Moreover, Lemma shows that 



(/'')o = ifji)^ so that '^"if'') = '^''i{f%) = '^"{(fd)'')- Thus, it is enough to prove 
Theorem 4.1 for maps of the form /g, that is for convex monotone homogeneous 
maps that are also multiplicatively homogeneous. 

We next reduce to the case where N'^(/'^) = {1,... ,7i}. Indeed, let N = 
{1, . . . ,n} \ N'^(/'') and consider g : M" ^ R" defined by g^{x) = f^{x) V x^ if 
i € N and gi{x) = fi{x) otherwise. Using the inclusion already proved in Corol- 
lary we get N'=(/) C N'=(/'') = {1, . . . , n} \ iV, hence '^"{g) is the disjoint union 
of '^'^{f) and of the set S£ of loops i ^ i with i £ N. This implies in particular 
that ^"{g)'' is the disjoint union of and ^, and that N^ig) = N'=(/) U N. 

Since 5(0) = and g'' > /'=, we get N'^ig'') D N'=(/'=) and '^"{g'') D '^"{f''). Again 
by Corollary ^ we get N'^(g'=) D N^^g) D N, hence, N'=(.g''j = {1, . . . Now 
if ^'=(5'') C ^'=(3)'=, then C ^'=(5'=) C ^''{g)'' = 'S^{ff U ^, and since 

N'=(/'^) = {1,... ,n} \ iV, we have "S^if^) C '^^{ff , which, together with the 
inclusion proved in Corollary 4.4, yields ^'^{^f^) = $^'^(/)'^. 

We are now reduced to show the following proposition, (Pn): for all convex 
monotone bihomogeneous maps f : M" — > R", and for all k > 1, such that N'^(/'^) = 
{1,... ,n}, we have C We shall prove (Pn) by induction on the 
dimension ??. Proposition (Pi) trivially holds, since in dimension 1. the only possible 
map / is the identity map. Assume now that (P^) holds for all dimensions £ < n, 
and consider a map / : R" R" as in (P„). 

Let F be an invariant critical class of / (as in Lemma 4.7), let iV = {1, . . . , n}\F, 
and put g = /at a? and h = fpF- Assume first that all the entries of g are finite. 
Then, by (|l^, ^'=(/) = ^"^(g) U ^'=(/i). Since F is also invariant for J*^, we also 

have ^'=(/'=) = '^"HT^n) U ^'=((/'=)ff). Using the first part of Lemma |j[ we 
obtain (/'^)vAr ~ fN ° f'^ ° iN ~ g'^ and {f^)FF = h*' . Moreover, if the entries of 'g 

g'', so ^%f'') = ^"(g'') U ^%h''). Thus, in order 



= o o 
are finite. Lemma 4.6 yields g'^ 



to prove '^^{f ) C '^^{f) , it is sufficient to prove the same inclusion for h and 'g 
and to prove that 'g has finite entries. 

Let us first prove 'i^^{h^) C 'i^'^{h)^ . Since F is an invariant critical class of /, 
^"^{h) is strongly connected and contains all the nodes of F. By Assertion ^ of Corol- 
lary 2.6, there exists P E dh{0) such that ^#'^(/i) ~ $^'(P), hence P is irreducible 
and '^"{h) = ^(P). Moreover, for aU Q <E dh{Q), the matrix belongs to dh{0) 
and is irreducible, so ^^(g) C ^(Q) C ^(^) = ^f(%^) C ^"(/i). Defining 
^(^) = Upg <5<.^(P) for all sets ^ of stochastic matrices, we get ^%h) = '^{dh{0)). 
Hence, ^'=(/i'=) c ^(a(/i'^)(0)) = ^(co((9/i(0))'=)) = ^((a/i(O))''0 C ^(9/i(0))'= = 
^'=(/i)'=. 

Let us now prove that all the entries of g are finite. This is indeed equivalent 
to the property 17(1) ~ 1, where 1 denotes the unit vector (which means that 
dg{0) n S'nn 7^ 0)- We know that g{l) < 1, so by monotonicity of g, g''{l) < 
■ ■ ■ < g{l) < 1. On the other hand, by Proposition 4.5, there exists a stochastic 
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(row) vector m with positive entries (i.e., the support of m is {1, . . .n\), such that 
mf^{x) > mx for all x G K". By Lemma 4.9, /oi^r — ijv°3, hence f^o\N = '^N°g^ 



Applying mf^{x) > mx to a; = iAr(2/), we obtain mNg^{y) > mNU, where mAr is 
the restriction of m to the set N {niN has positive entries since all the entries 
of m are positive). So mwl > ruNgiX) ^ ' ' ^ '^Ng^{^) > mjvl. This yields 
toatI = mj^qiV). Using the positivity of ttiat and the inequality g(l) < 1, we get 

1 -<?(!)■ 

We finally show that '^'^[g^) C '^^{g)*' , by using the induction assumption. In- 
deed, since F 7^ 0, the cardinality of iV, which is the dimension of the space on 
which g operates, is strictly less than n. Moreover, since niMg^iy) > TriNU for all 
y E M.^ , we deduce that rriNg^iy) > rnf^y. Since 5 has finite entries, ~ g'', so 



that m^g {y) > for all y £ R , and Proposition 4.5 shows that N'^{g ) = N 



The induction assumption can then be applied to g, and yields '^'^{g^) C $f'^(g)'^. 
This concludes the proof of (P„). □ 

5. Cyclicity Theorem for Convex Monotone Homogeneous Maps 

In this section, we use our knowledge of the eigenspace of / to study the asymp- 
totic behavior of when k tends to -foo. In particular, we are interested in the 
periodic orbits of /, which are of the form {f^{xy\keM^ with /'^(x) = x for some 
c > 1. The set of such c is exactly the set of multiples of a positive integer, which 
is the length of the orbit {f''{x)}km- 

Let us first recall some more or less classical facts on periodic orbits of stochastic 
matrices. The cyclicity c(^) of a strongly connected graph ^ is the gcd of the 
lengths of the circuits of The cyclicity of a graph with strongly connected 
component ^^i, ... is defined by c(^^) = lcm(c(^^i), . . . , c{'^s))- For a stochastic 
matrix P, we define the cyclicity of P as: c(P) = c{^^{P)). The name "cyclicity" 
is justified by the following result. 

Proposition 5.1. If P is a stochastic matrix, then, the length of any periodic orbit 
of X ^ Px divides c{P). 

Proof. If P is a stochastic matrix with cyclicity c = c(P), then the Perron- Frobenius 
theorem shows that P^'^ — *■ 11 when t ^ 00, where 11 is the spectral projector of 
for the eigenvalue 1. If P^x = x, then, x = Hmf_>tx) pi'=™('=''=)^x = Ila;. Since the 
spectral projector satisfies P'^II = 11, we get x = P'^x, which shows that the length 
of any periodic orbit of P divides c. □ 

More generally, we define the cyclicity c{f) of a convex monotone homogeneous 
map / as the cyclicity of its critical graph c{1^^{f)). 

Proposition 5.2. Let f denote a convex monotone homogeneous map that has an 
eigenvector v. Then, 

(18) c(/) = gcd{c(P) I P G df{v), Nf (P) = N=(/)} . 

Moreover, there exists P £ df{v) such that N^(P) = N'^(/) and c(P) = c(/). 

Proof Let P e df{v) such that Nf(P) = N'=(/). Since ^^(P) C ^"(/), and since 
both graphs have the same set of nodes, each critical class C of / can be partitioned 
into final classes of P, C = Pi U . . . U P^, and (P)|c C ^'=(/)|c, where 
denotes the restriction of the graph to C. Hence, the cyclicity of ^^'^(/)|c, which 
is equal to the gcd of the lengths of circuits of '^'^{f)\c divides c(^^^(P)|c) = 
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lcm(c(Pp^i?j ), . . . ,c{PF^,Fr.))- Since this divisibility holds for all critical classes C 
of /, c(/) divides c(P). Conversely, Assertion]^ of Corollary 2.6, there is a matrix 
P G df lv) such that (P) = which shows that the right hand side of ^ 

divides c(/). □ 



Proposition 5.3. Let f be a convex monotone homogeneous map that has a fixed 
point and let^i, ... , $^<, be the strongly connected components of^'^{f). Then, /'^^■^' 
has c(?#i) + • • • + c(^^s) critical classes and has cyclicity 1. 



Proof. From Theorem we get '^"(f^^^) ^ ^%fy^f'> ^ U^i^-'^^' (disjoint 
union). For any strongly connected graph with cyclicity c and any integer k, '^^'^ 
has c strongly connected components with cyclicity 1 (this is a well known result 
which follows from the decomposition of 
Chapter 6, §3] and |BP94 Chapter 2, §2]). 
^'^(/'^'■■^■') has c(?#i) + - • -+0(^3) strongly connected components with cyclicity 1. □ 



§ into cyclic classes, see e.g. [ |KSK66 
Since c(^^i) divides c(/), we obtain that 



We shall also need the following classical fact. 

Proposition 5.4. A monotone homogeneous map 
orbit has a fixed point. 



which has a periodic 



Proof. If / has a periodic orbit, there is an x G K" and fc > 1 such that /*''(x) — x. 
Consider z = a; A f{x) A ... A /'""^(cc). By monotonicity of /, f{z) < f{x) A /^(x) A 
...A f''{x) = z. Thus, the sequence {/^(z)}fgN is nonincreasing, and since by 
nonexpansiveness of /, \f^{z) — f^{x)\ < \z~ x\ is bounded, f^{z) which is bounded 
and nonincreasing converges to a limit which is an eigenvector of /. □ 



Theorem 5.5. The length of any periodic orbit of a convex monotone homogeneous 
map f divides c(/). 

Proof. If / has a periodic orbit \ f'' (x)} keN , then, by Proposition 5.4, / has a fixed 
point V. It suffices to prove Theorem |5.5| when c(/) = 1. Indeed, if c(/) ^ 1, con- 
sider g = f'^^^\ which is such that c{g) = 1 (by Proposition 5.3) and {g''{x)}k£n 
is periodic. If we know that Theorem 5.5 holds for g, we get x = g(x) = f'^'^^'>{x), 
hence the length of the periodic orbit \j{x)}keN divides c(/), which yields Theo- 
rem ^.5| in the general case. 

Let us now prove the theorem when c(/) = 1. Using Proposition |5.2| , we can 
find a matrix P e df{v) such that c(P) c(/) = 1 and N^{P) ^ N%f). Using 
f{v) — V and f{y) ~ f{v) > P{y — v), we get after an immediate induction, 

(19) f''{x)>P''{x-v)+vyken. 

In particular, if c is the length of the orbit {f''{x)}kefii, x = f'^{x) > P'^{x — «)+«. 

To make the proof more intuitive, we shall first show the theorem when N'^(/) = 
{1, . . . , n}. By Proposition 4.2, P^ again has {1, ... , n} as union of final classes, 
and applying Lemma 2.9 to a; — w > P'^{x — v), we get x — v 
has cyclicity 1, this implies that P{x — v) ~ x ~ v, and using 
By monotonicity of /, we get x = fix) > • • • > f{x) > x. 
c=l=c(/). 

Consider now the general case where F = N'^(/) 7^ {1, 
x — v > P'^{x 
on Nf(P'=) = 



= P''{x-v). Since P 
|l|), we get f{x) > X. 
Thus, f{x) — X, and 



n}. As before, we get 

By Lemma 2.9 and Proposition 4.2, this yields x — v = P'^{x — v) 

71} \ P, this equation can 



Nf (P) = P. Since Pfn = for = {1 
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be rewritten as {x — v)f ~ {PffY'{x — v)f- Since c{Pff) = c(P) = c(/) = 1, we 
deduce {x — v)f = Pff{x — v)f- Then, for all fc > 1, {Pff)''{x — v)f ~ {x — v)f 
or equivalently {P^{x — v))f = {x — v)f- Putting this in (p^), we get f^{x) > x 
on F. Consider now z — x A f{x) A • • • A /"^^^(x), which coincides with x on F. By 
monotonicity of /, f{z) < z. Hence, by Lemma |3.3| , w = f^iz) is a fixed point of 
/ and it coincides with z on N'^(/) = F. Since f{w) = w, we get /'^(w) = it). So 
w and X are two fixed points of f', which coincide on F = N'^(/) = N'^(/^). This 
implies that w = a; by Theorem 3.4. Hence, f{x) = x, and by the minimality of c 
(as the length of the orbit of x), we get c = 1. □ 



Corollary 5.6. The lengths of periodic orbits of convex monotone homogeneous 
maps R" M" are exactly the orders of the elements of the symmetric group on n 
letters. □ 



Combining Theorem 5.5 with the theorem of Nussbaum [Nus90 and Sine |Sin90|, 
we get: 

Corollary 5.7. /// is a convex monotone homogeneous map M" R" with eigen- 
value A, then, for all x G M", f'"^^^^(x) — kc(f )X converges when k oo. □ 

In other words, introducing the normalized map g = f — X, which is such that 
g*^ — f^ ~ kX, we see that any orbit {g''{x)}k<£fi of 9 converges to a periodic 
orbit of length at most c{g) = c(/). We also get, as an immediate consequence of 



Theorem 5.5 



Corollary 5.8. For any convex monotone homogeneous map f that has a fixed 
point, the union of the periodic orbits of f is exactly the set of fixed points 

off<i\ □ 



4.1 



and Theorem 3.4 



Combining Corollary 5.8, Proposition 5.3, Theorem 
obtain: 

Corollary 5.9. Let f, ^i, . . . be as in Proposition |5.4 and let C ~ N'^(/). 

Then, rc sends bijectively the union of periodic orbits, S'{f'^^^'^), to a convex inf- 
subsemilattice ofM.^, whose dimension is at most equal to m{f'^^^'^) ~ c(^#i) + • • • + 
c(%). ' □ 



6. PlECEWISE AfFINE CONVEX MONOTONE HOMOGENEOUS MAPS 



6.1. Dimension of the Eigenspace. As discussed in § |1.2| , the dimension of the 
eigenspace of a piecewisc affinc convex monotone homogeneous map was charac- 
terized by Romanovsky |Rom73 | and by Schweitzer and Federgruen |SF78 |: this 
shows that the bound on the dimension given in Corollary 3.6, is attained when / 
is piecewise affine. In this subsection, we give an independent proof of this fact, 
which shows some qualitative properties of (ff{f) (connection between <^'(/) and 
S'{fy), for any eigenvector v, role of invariant critical classes). 

Let us first recall some basic definitions and facts. A polyhedron is an intersection 
of finitely many half-spaces. A map / : M" ^ K is piecewise affine if M" can be 
covered by finitely many polyhedra such that the restriction of / to each polyhedron 
is affine. The following result is an immediate consequence of classical results on 
convex maps with polyhedral epigraphs [Roc7C|. 
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Proposition 6.1. A convex map f : K" R is piecewise affine if, and only if, 
there exists a finite set C dom /* such that 

(20) /(x)=max(p.x-r(p)) . 

It will be useful to consider bihomogeneous maps. 

Corollary 6.2. A piecewise affine convex monotone bihomogeneous map f : M" 
M can be written as 

(21) f{x)=inaxp-x , 
where ^ is a finite set of stochastic vectors. 



Proof. By Proposition 2A, dom/* C therefore, the finite set in (|20| ) is 
composed of stochastic vectors, and since /* takes only the values and +cx) when 
/ is multiplicatively homogeneous, we get (^. □ 

The following simple fact shows that eigenspaces of convex piecewise affine maps 
can be effectively computed. 

Corollary 6.3. The eigenspace of a convex piecewise affine monotone homoge- 
neous map f : M" — > R" is a finite union of polyhedra. 

Proo f. W e a ssum e, without loss of generality, that / has eigenvalue 0. By Proposi- 
tions 2.1 and 6.1, we can write each coordinate of / as fi{x) — maxpg ^. {p-x~f*{p)), 
where is a finite subset of To any i G {1, . . . , n} and p € we associate 
the (possibly empty) polyhedron Ki^p — {x € R" | Xi = p ■ x — f * (p) > q ■ x — 
f*{q)yq e ^i}. We denote by $ the set of maps ip: {1,... , n} ^ U . . . U ^„ 
such that ip{i) €E for all i G {1, . . . ,n}. To lyS G $, wc associate the polyhedron 
Kip = f^i<i<nK.i ip(^iy If x = f{x), then, for each i, by finiteness of we have 
Xi = maxpg^; p ■ X ~ fi{p) ~ v{i) ■ x — f*{ip{i)), for some Lp(i) G This shows 
that (?(/) C Uipg^iTi^. Since the other inclusion holds trivially <S{f ) = Uipg^X;^ is 
a finite union of polyhedra. □ 

The proof of Corollary |6.3| yields an algorithm (with exponential execution time) 
to compute ^(/), but it does not tell much about the geometry of S{f). Wc shall 
next prove more qualitative properties. 

The following observation shows that the directional derivative defined in 
Section ^ is a "tangent" map of /. 

Lemma 6.4. /// : R" —tM. is a piecewise affine convex map, then, for all v G R", 
there is a neighborhood V of v such that 

(22) fix)^f{v) + fUx-v) VxGl/. 

Proof. The definition of df{v) together with (|l^) show that the inequality > holds 
in (^). To show the other inequality, we write / as (po|), and set ^ = ^ fl dfiy), 
^ = ^ \ =2. We can write f ^ gV h where 

(23) g{x) = max (p ■ x ~ f*{p)) = f(v) + maxp ■ {x - v) 

p€.^ndf{v) pes 

(since /(t;) ~ p-v — f*{p), forallpG df{v)), and /i(a;) = maxpg^(p-a; — /*(p)). The 
functions g and h are continuous and satisfy /(f) = g{v) > h{v) = maxpg^\aj(^) (p- 
V — f*{v)), where the strict inequality follows from the characterization (H) of df{v) 
and from the finiteness of Therefore, there is a neighborhood V of v such that 



26 



MARIANNE AKIAN AND STEPHANE GAUBERT 



f{x) = g{x) V h{x) — g{x) for all x G V. Combining this with (|2^) and (|lO|), we get 
f ix) ^ glx) < f{v) + f^{x - v) for all xeV. □ 

Our interest in stems from the following reduction. 

Lemma 6.5. Let f be a piecewise ajjine convex monotone homogeneous map ffi" — > 
K" with eigenvector v. Then, the dimensions of S'(f) and 'S'{fy) are the same. 

Proof. We first note that by construction of , the critical classes for / and are 
the same. Let C = N'=(/) = N'=(/^). By Corollary ^ there is a set [/ C (/^) 
such that rc{U) is a convex set of dimension dim ( f (//,). Of course, we may choose 
a bounded U. Then, picking V as in Lemma |6.4| , we get v + eU C V, for e small 
enough, and applying (p2[), we get 

f{v + eu) = f{v) + f[,{eu) ^\ + v + €u MueU , 

which shows that v + eU C Therefore, Aim<g{f) > dimfr(/;;). To show 

the other inequality, we take a set U' C <^'(/) such that rc{U') is a convex set of 
dimension dim(f(/). Then, by convexity of S^{f), for all < e < 1, r(7((l — f)v + 
eC/') C S^if). We may choose a bounded [/'. Then, taking Ve = rc\rc((l - e)w + 
eC/')) C <^'(/) and using the contin uity of r^^ from to (which follows 



from the last assertion of Theorem 3^), we get Ve C V for e small enough, and 
using ( p2[ ) again, we get, for all w G Ve, 

w - w = /(■!«) - A - iJ = /(ij) + - ij) - A - w = f'^,{w - v) , 

which shows that V.-vC S'if^). Therefore, dim^(/,',) > dim^(/). □ 

We next give a new proof of the following result, equivalent forms of which were 
proved by Romanovsky, and Schweitzer and Federgruen. 



Theorem 6.6 (Cf. [ lRom73| , Th. 3], ^F78| , Th. 5.1 and Th. 5.3]). The dimen 



ision 



of the eigenspace of a piecewise affine convex monotone homogeneous map f is 
equal to the number of critical classes, m(/). 

We shall reduce to the special case of maps g > I with eigenvalue (where / 
denotes the identity map). This case is simpler because, when g > /, 

(24) (.9) = {x G R" I g{x) ^ x} = {x G R" | 5(2;) < x} = ^+{g) . 

Lemma 6.7. If g : R" — > R" is a piecewise affine convex monotone bihomogeneous 
map such that g > I, then, S'ig) is a convex set of dimension 1x1(17). 



Before proving Lemma |6.7| , let us show that it implies Theorem 6^. Let v 
denote any eigenvector of /, and consider the maps and g ~ f^ V I, which 
both are piecewise affine, convex, monotone and bihomogeneous. Let s = rn(/^), 
let Ci, . . . ,Cs denote the critical classes of fy (which are also those of /), and 
C = W{fy) = Ci U . . . U Cs. It is immediate to check that g also has Ci, . . . ,Cs 
as critical classes, and that is has additional critical classes of the form {i}, where 
i is any element of the complement N = {1, ... ,n} \C. Thus, m(g) = s + \N\. 



Applying Lemma 6.7 to g, we get that dimS'{g) = s + |A^|. Let S denote any section 



of f'y (recall that sections were defined before Corollary |3.6| ) . Then, S' = S U N 



is a section of g, and, by Corollary 3.6, fs'i'^ig)) is a convex set of dimension 



s + \N\ = |5"|, which implies that fsi'^id)) is a convex set of dimension s. 



Using Lemma 3.3, we see that (/',)" sends <f(g) = S'^{fy) to ^(/^), and that 



fsi'^id)) coincides with fsi^if'v)) {S C C). Therefore, rs(<^(/i)) has dimension s, 
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which imphcs, by CoroUary 3.6, that dim<#'(/,y) = s = m(/,y). By Lemma 3.5 



get dimS'lf) = dimff(/^) = m(/,',) = m(/). which shows Theorem 3.6. □ 



we 



Proof of Lemma \6. % The convexity of (S {g) foUows from (^4|) . We shall prove that 
dim. <S{g) = m{g) by induction on the dimension n. (The constructions of the proof 
are illustrated in Example 6.9 below). 

If n = 1, g is the identity map, then, dimS[g) = m{g) = 1, and Lemma |6?7| holds 
trivially. Let us now assume that n > 2. Denote by F an invariant critical class 
of g, as defined in Lemma 4.7, let N = {1, ... ,71} \ i^, and put h = (Inn, where 
Qnn ~ <'n ° S ° ''Ni with the notation of Section ^ Since g > /, we have gNN ^ I, 
so h > I. 

Corollary |6.2| allows us to write for each 1 < i < n, gi{x) = maxpg^^ p ■ x, where 
C dgi{0) is a finite set of stochastic vectors. Consider now, for all i E N, the 
partition = ^ U where ^ = {p e | = 0, Vj S F}, and = 3^., \ J^. 
We have hi[y) = max^g^. p • \N{y) for alH G TV, y G (in particular, ^ 7^ 0, 
for all i E N, because h > I takes finite values), and, for all x G M", 

(25a) 9i{x) = hi o rN{x) y maxp ■ X \/i E N, 

(25b) g,{x) = {gFF°rF{x))i £ F . 

From dl^), we get the disjoint union ^^=(5) = ^'^(/i) U^'=(5ff), so m{g) = m{h) + \. 
By the induction assumption, S{h) is a convex set of dimension m(ft,), which implies 
that we can find a bounded convex set U C <S{h) of dimension m{h). We shall 
complete the elements of U to get eigenvectors of g as follows. To each u E U, 
and A G M, associate the vector v{X,u) G M" such that rN{v{\,u)) — u, and 
ri?(u(A, u)) ~ A (the constant vector). We get from ( |25b| ) and Lemma |4j| , 

(26) g,(w(u. A)) = (gpFUX) = A = v{u, A), E F , 



and we get from (25a), 

(27) gi{v{u, A)) ~ hi{u) V maxp • v(u, A) = V maxp • v{u, A) Vi E N . 

Since for all i G iV and p E Pj ^ Q for some j E F, we get p ■ v{u,X) = 
^j^N Pj'^j ^j^pPj^ ^ when A —00. Since U is bounded and the sets 
are finite, the limit is uniform in m G J7 and p E J^i- Therefore, we get from (27) 
that 

(28) g^{v{u, A)) {viu, A)), Vt G iV , 

holds for all It G J7 and A < Aq, for some Aq G M. Combining (^6|) with (|2|), we 
get that g{v{u, A)) = v{u, A) for all u E U and A < Aq, which shows that S'{g) has 
dimension m{h) + 1 = m(g). □ 

We obtain in passing some information on the restriction of <S'{g) to the set of 
invariant critical classes. If / C {1, . . . , n}, we denote by 1/ G M" the vector such 
that (l/)i = 1 if z G /, and (l/)i = otherwise. 

Corollary 6.8. Let g denote a convex piecewise affine monotone bihomogeneous 
map with invariant critical classes Fi,... ,Fr. Then, the restriction of S'{g) to 
FiU---(JFr is equal toRlpi^ h Ml f,, . 

Proof. The reduction in the proof of Theorem |6.6| shows that we can assume that 
g > I, so that we are in the situation of Lemma |6.7] . The corollary is obtained 
by a straightforward variant of the induction argument of Lemma 6.7, in which F 
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is replaced by i^i U • • • U Fr, and A) is replaced by the vector v(u, Ai, . . . , A^) 



defined by rjv(w(u, Ai, . . 
vector), with Ai, . . . , Ar 
maxi Ai close enough to 
A e M, v{X+u, A+Ai, . . . 
can take any value in W 
contains Ml^i H h M: 



iv{u,Xi 



, Xr)) — Xi (the constant 



Theorem 3.4 



. , Ar)) — u and rp- 
G R, and u G U. The proof of Lemma 6.7 shows that for 
—oo, v(u, Ai, . . . , A, ) is an eigenvector of g. Then, for all 
, A+Ar) is an eigenvector of and since (A+Ai, . . . , A+A,.) 

this implies that the restriction oi to Fi U • • • U F,. 
.p^. The other inclusion follows from the first assertion of 

□ 



6.2. Examples. 

Example 6.9. Let 01,02,03 > 0, and consider 

/ xiV {-ai + {x2+X3)/2) 
/ : R3 ^ fix) = X2 V (-02 + (xi + X3)/2) 

V X3 V (-03 + (:ri +a-2)/2) 

We have /(O) = and /o(a;) = x. Therefore, Theorem |6.6| states that (a{f) has 
dimension 3. Indeed, it is immediate to check that = {a; | Xi > —ai + (xk + 

Xj)/2, VI < i < 3, fc 7^ i, j 7^ i}, and this set has dimension 3. Since fS'(/) is invariant 
by the translations of vector (A, A, A), for all A S M, it is convenient to represent 
the projection of S'{f) on any plane orthogonal to the direction (1, 1, 1), which is 
as follows (we take oi = 1/2,02 = 1,03 = 2, the point (0,0,0) is represented by a 
bold point, the projection of the eigenspace is the shaded region). 

2:3 



Xl 



"X2 



By deforming this picture, it should be obvious that in the limit case oi = 02 
0,03 > 0, the eigenspace (?(/), which looks as follows. 



2^3 



Xl 



'X2 



has still dimension 3. Let us check this. We have in this case 



(29) 



Xl V (.X2 + X3)/2 
/^(x) = ( X2 V (Xl +X3)/2 
X3 
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We see that /q has the three critical classes {1},{2},{3}, and, by Theorem 6.6 
S'{f) has dimension 3. It is instructive to illustrate the proof of Lemma ^^by the 
example of 5 = /q. There is only one invariant critical class, F = {3}. We have 
— {1,2}, and h{y) = y, for all y e M?. The representation ( p9| ) corresponds 
to^i = {(1,0,0)1,^1 = (0,1/2,1/2), ^2 = {(0,1,0)}, ^2 = (1/2,0,1/2). Any 
u = (ui,U2) € is an eigenvector of the identity map h. The last step of the 
proof of the lemma shows that for A negative enough, v{u,X) = (ui,M2,A) is an 
eigenvector of /g. More precisely, it is not difficult to see that the eigenspace of /g 
looks as follows 

X3 



where the shaded region now extends infinitely in the direction X3 —>■ —00. (In this 
example, <o'(/g) is the contingent cone to S'(f) at the point 0.) 



Example 6.10. We next present a variant of Example 6.9, for which / is of cyclicity 
2. Let oi, 02, 03 > 0, and 



X2V {-ai + {xi+X3)/2) 
/ : M-^ R3, fix) = I xi V (-02 + ix2 + X3)/2) 

X3\/ {-a3 + ixi+X2)/2) 

We have /(O) = 0, and (9/(0) = {/'(O)} with 

1 
/'(O) =(100 
1 



which gives the critical graph: 



as 



Therefore, '^^{f) is the union of two strongly connected components. and $#2, 
with respective sets of nodes {1,2} and {3} and cyclicities 2 and 1, which implies 
that / has cyclicity c(/) = lcm(2, 1) = 2. An immediate computation shows that 



S{f) = {x e I a'i + xz>xi=X2> -2(ai A 02) + X3} 



This two dimensional convex set is represented by the vertical bold segment on the 
following figure: 
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X3 



/(,'/) 



Xi • 



■X2 



Corollary |3.8| says that the union of periodic orbits of / is equal to <o'(/^). A new 
computation shows that S'{f'^) is equal to the triangular region determined by the 
inequalities 



xi > -(ai A 02) + (2:2 + a;3)/2 , 
X2 > ~{ai A a2) + {xi + X3)/2 , 
X3 > -as + {xi + X2)/2 , 



which is depicted in the figure. On this example, Theorem 6.6 predicts that (S'{f) 
is of dimension 2, and Proposition ^.3| , combined with Theorem 6.6, predicts that 
^{P) is of dimension c(J^i) + c({#2) = 3. It is easy to see that / acts on as 
the reflection with respect to the plane xi = X2. As an illustration, a periodic orbit 
{y, /(y)} is depicted on the figure. 



Example 6.11. Consider any convex monotone homogeneous map / : R" R", 
whose eigenspace has dimension n. We show here how, in that case. Theorem 3.6 
allows us to characterize the set We assume without loss of generality, that 

the eigenvalue is 0. Since the dimension of #(/) is n, the number of critical classes 
must be equal to the dimension, n, hence / > /, or equivalently, f ~ I\J g, for some 
convex monotone homogeneous map g. Then, 



(30) 



S{f) = {x e M" I x>Px- g*{p) VP e dom.g*} 



Half-spaces of the form Xi > PiX — gf{p) involved in this definition are special. 
Indeed, the later inequality can be rewritten as 



(31) 



Xi > 



with < aj, 7 € M, and J2j=^i o^j = 1- Using ( ^ ) and (|3l|), we get that a subset of 
R" of non-empty interior is the eigenspace of a convex monotone homogeneous map 
/ : R" — !■ R" if, and only if, it is the intersection of half-spaces of the form (^l]). 
This implies, for instance, that the region at the left hand side of the following 
figure is the eigenspace of a convex monotone homogeneous map: 
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X3 



a K h 



a V b 



, A h 



Xl- - -0 ■X2 Xl- 'X2 

a V6 

/ convex / concave 

The fact that S'{f) is an inf-subsemilattice of M" (but not a sup-subsemilattice) 
is ilhistrated by the four points a,b,a\/ b,a Ab. Eigenspaces of concave monotone 
homogeneous maps have symmetric shapes, hke the one at the right hand side of 
the figure. 

Example 6.12. The following example shows that for a non-convex picccwise affine 
monotone homogeneous map /, the dimension of the eigenspace is ill-defined. Con- 
sider / : R3 ^ M^^ 

/ {xi A a;2 A X3) V {xi A (-1 + X2) A (f + X3)) 
(32) fix) = {2 + Xl) /\ X2 A (3 + X3) 

\ {Xi/\X2AX3) 

Then, it is easy to check that f = f^, and that the eigenspace of / is the following 
flag shaped set: 

X3 



Xl 



•X2 



Explicitly, (f(/) A'l U K2, where Ki = {(A, A + t,X) \ A G R,0 < t < f}, and 
K2 = {(A, A + t, A + s) I A e R, 1 < t < 2, -f < s < 0}, which shows that the 
"local dimension" of the eigenspace near a point x G S'{f) is 2 (if x is in the relative 
interior of Ki) or 3 (if x is in the interior of K2). Although we do not need this here, 
let us mention that there is a systematic technology to build such examples, which 
originates from max-plus algebra: / is a projector on the max-plus semimodule 
generated by the columns of the matrix 


1 2 
-1 -1 



See |CGQ97i for details 



6.3. Computing $^'^(/). To conclude this section, we give a polynomial time al- 
gorithm to compute 

(33) #(co(^i X • • • X ^„)) 
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given finite sets of stocliastic vectors ^i,... ,^„. Tliis algorithm allows us. in 
particular, to compute the critical graph of a piecewise affinc convex monotone 
homogeneous map /, provided that an eigenvector u G M" of / is known. Indeed, 
the coordinates of / are of the form 

(34) /,(x) = max(p.a;-/;(p)) , 

where the arc finite sets of stoc hastic v ectors, and, setting J^i = {p G \ 
P-u-f*{p) = ft{u)}, it follows from ^W9S| , Th. 10.31] that dfi{u) = coi2,, hence 
df{u) = co(.^i X • • • X ^n)- Since '^^Hf) ~ '^^{df{u)), the problem of computing 



'i^'^if) reduces to that of computing (33) 



To write the algorithm, it will be convenient to consider more generally a finite 
family where C is a finite set of suhstochastic vectors. For any 

rectangular set =S of suhstochastic matrices, we define ^^^(^) as the union of the 
graphs of the matrices Pff, where P £ F is a final class of P, and Ppp is a 
stochastic matrix (this definition is consistent with the one of ^#'^(/) for a monotone 



subhomogeneous map /, that we gave in §1.4). 

The algorithm can be specified as a recursive function, with input 
and output $#^(co x^g/^i). The function first builds, for all i E I, the subset 
C of row vectors with row sum 1, together with the graph G = ^(Xig/=S-) = 
U_PGXie7^'^(^')- If all the strongly connected components of G are trivial (we say 
that a strongly connected component is trivial if it has only one node and no arcs), 
the function returns the empty graph (with no nodes). Otherwise, we proceed as 
follows. We denote by F the union of final classes of G, and put N ^ I \ F. (The 
set F is indeed the union of invariant final classes of / when co ~ df{u), 



see Lemma 4/7.) For i G iV, we define the sets C of row vectors obtained 
by restricting to N the vectors p & £2[ such that pj = 0, for all j E F. We denote 
by G\f the restriction of G to F. The identity 

(35) ^f(cox,e/^,) ==G|i.U^f(cox,ejv^f) , 



yields a recursive algorithm to compute ^^^(co Xi^jJ^i). (The identity (35) is similar 
to (0).) 



Applying this algorithm to the case of Example 2.4, we get =Si = 
{(1, 0, 0), (1/2, 1/2, 0), (1/2, 0, 1/2)}, ^2 = {(0, 1, 0), (2/3, 1/3, 0)}, ^3 = {(0, 0, 1)}, 
J^l = F = {3}, = {(1,0), (1/2, 1/2)}, ^'i = {(0, 1), (2/3, 1/3)}, so that (H 
shows that $#^(co(^i x x =^3)) is the union of a loop at the node 3, and of the 
complete graph with nodes 1,2. 

7. Stochastic Control Interpretation 

In this section, we briefly explain how the above results can be applied to sto- 
chastic control. This application also makes the results more intuitive. See for 



instance | Whi86 |, or [ ELL96 for more background on stochastic control. 

A Markov control model with state space {1,... ,71} is a 4-uple 
[A, {Ai}i<i<„, {ri}i<j<„, {Pi}i<i<n), where: A is a set, called action 
space] for each state 1 < i < n, is a nonempty subset of A, whose elements 
are interpreted as possible actions] ri is a map from Ai to R, the image r° of a 
is interpreted as an instantaneous reward received when action a is performed in 
state i] Pi is a map from Ai to the set S^n of stochastic vectors, and the j-th entry, 
Pij , of the image P° of a is interpreted as the transition probability from state i to 
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state j, when the action a is performed. It will be convenient to assume that A is 
a topological space equipped with its Borel a-algcbra. that the Ai are Borel sets, 
that the maps and Pi are measurable, and that 

(36) for all a; € M", sup {r^ + P^x) is attained (and finite) 

aeAi 

(this is the case, in particular, if Ai is compact, is upper semi-continuous, and 
Pi is continuous). 

The intuitive notion of strategy, i.e. of causal rule telling which action to 
choose, is captured by the following definitions. An history is a sequence of 
states (zo,«i,...), a partial history is a finite sequence (ig,... ,jfe). A (random- 
ized) strategy is a sequence 7 = (70, 71, ... ) where 7fc is a map which to a partial 
history (ig, . . . ,ik) associates a probability measure on A such that Ai^ 

has probability 1, i.e. 7^"'"' '''' [^ifc] = 1: the action at time k, Ok S Ai,,, will 
be chosen with probability ^]?'"''^''. We say that 7 is deterministic if 
is a Dirac measure. We say that 7 is Markovian if 7^"' ' '*'' only depends of fc 
and ife, and that a Markovian 7 is stationary if 7^ is independent of /c. Mar- 
kovian stationary policies are obtained by choosing, for all 1 < z < n, a prob- 
ability measure t; on A such that Ti\Ai] = 1, and taking 7^° = t^^. We 
denote by t°° the (Markovian, stationary) strategy 7 built in this way. For any 
strategy 7, and initial state io, we consider a (state, action) stochastic process 
(^Oj ao)j cki), . . . with values in {1, . . . , n} x A, such that = the law of ak 
knowing ^0, ■ • ■ 7 ao G ^Co' • ■ • > £ ^Ci-i is equal to 1^''"' , and the law of 
^fc knowing $0, • • ■ ,6-i,ao S ^Co' ■ ■ ■ jCkfe-i G ^j^.i is equal to P^^!'S^\^- When 7 
is Markovian and stationary, S.k simply becomes a time homogeneous Markov chain 
with initial state i and transition matrix P'^ , 



P^= f Pt,dn{a) 

J A 



The ergodic control problem consists in finding a strategy 7 which maximizes, 
for all initial states \ < i < the mean reward per time unit: 

(37a) m7 = liminf —K'-Hr'i'' +■■■ + r""-M . 

We denote by 

(37b) fii = sup fi] 

1 

the optimum mean reward. 

A closely related problem is to solve, for large N , the horizon N problem, which 
consists in maximizing 

(38a) vJ{N) = E^-'tC + ' ' ' + C-" + <^««) ' 

where the final reward is a map from {!,... , n} to R, or equivalently an element 
of R". We set 

(39) w,(iV) =supz;7(7V) . 

7 

The vector v{N) G M" is called the value function. When 7 — t°° , we shall simply 
write fi'^ and w'^, instead of /x"^ and v'^ , respectively. 
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The study of both the ergodic control and finite horizon problems relies on the 
dynamic programming operator, or Hamiltonian, f : M" K". 

(40) f,ix) = sup{rt + P^-x) . 

a£Ai 



It is obvious from ( [40[ ) (and well known, see e.g. [Whi86, Chap. 22, Th. 6.1]), 
that the map / is monotone, (additively) homogeneous, and convex. Conversely, 
the Legendre-Fenchel duality theorem shows that a convex monotone homogeneous 
map / can be written as 

(41) /,(x) = sup {p.x-f*ip)) . 



Since dom/* is included in the set of stochastic vectors (Proposition 2.1), and 
since p i~> f*{p) is lower semi-continuous, this is clearly of the form (^ ) (although 
dom/* need not be compact, note that property ( |36| ) is satisfied). Moreover, when 
the action space A is finite, dom /* = co Ai is a polyhedron, and, by Proposition |6.l| , 
fi is piecewise afhne. Therefore: 

Proposition 7.1. The dynamic programming operators of Markov control models 
with state space {1,... ,n} are exactly the convex, monotone homogeneous maps 
K" —^ R". Moreover, Markov control models with finite action spaces correspond 
to piecewise affine maps. □ 

(The representation (^ij) provides of canonical form for a Markov control models, 
in which one can choose, when in state i, the transition probability from state i, 
pG dom/*.) 

The value function v(N) can be computed recursively via the dynamic program- 
ming equation 

(42) v{0) = (j>, viN) = f{v{N -1)) 



(see e.g. [HLL96, Th. 3.2.1]). Therefore, an eigenvector u, with associated eigen- 
value A, yields a stationary solution of the dynamic programming equation (^2[), 
v{N) = NX + u, corresponding to the final reward cj) = u. Such stationary solutions 
are of economic interest. Indeed, when f{u) = A + u, we set 

(43) A,^{a(^A,\ A + u, = rf + P>} 

{Ai 7^ thanks to (|3^)) and build a Markovian stationary strategy r°° by picking 
any probability measure r such that r^Ai] ^ 1. A standard resu lt, that we shall not 
prove (resuhs of this kind can be found in ]lWhi8^ , Chap. 31] or llHLL96| , Th. 5.2.4]), 



states that such a strategy is optimal both for the ergodic control problem and for all 
the finite horizon problems with final reward (j) = u, which means, loosely speaking, 
that taking (j) ^ u makes it possible for the player to behave (optimally) in the short 
term as he would in the long term (see [ YK92t for more details on the economic 



interpretation) . 

Another motivation for describing the set of eigenvectors comes from (^2|): Corol- 
lary t).7| shows that 



v{Nc{f)) — Nc{f)\ converges, when N oo, 
and the possible values of the limit are precisely the eigenvectors of f'^'^^^ 
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Critical classes also have a stochastic control interpretation. If 7 = r°° is a 
Markovian stationary strategy, we define the vector r"^ G (R U {—00})" by 

r°;dn{a) . 



A 

If is a final class of P"^, we denote by nip the unique invariant measure of the 
F X F submatrix of P"^, and by € the restriction of the vector r"^ to F. 

Proposition 7.2. Let us assume that the dynamic programming operator f has an 
eigenvector u, with associated eigenvalue A. Then, 

(44) A = max — max /i[ and max fij — max m'^rp , 

, l<j<)i F final class of 

where the first two max are taken over randomized strategies 7, and randomized 
Markovian stationary strategies r°° , respectively. Moreover, if all the Ai are com- 
pact, and all the maps ri and Pi are continuous, the elements of'^'^{f) are precisely 
the F such that mprp — A. 

Thus, the critical classes of / are exactly the maximal final classes of randomized 
Markovian stationary strategies that are optimal for the ergodic control problem. 
(Considering randomized strategies is essential for this equivalence to hold, even if 
the maximum in (^) is also attained by deterministic strategies.) In particular, 
the critical nodes, i.e. the nodes of '^^{f), are exactly the nodes which are visited 
infinitely often, almost surely, by the trajectory of at least one optimal Markovian 
stationary policy. 

Proof. T he equa lities in ( ^4|) are standard results of stochastic control (see for 



instance |IILL9£, Th. 5.2.4] for the first two equalities; the last equality follows 



from the ergodic theorem for reducible Markov chains, see for instance | Whi86 
Ch. 31, §6]). We will only prove here the characterization of critical classes. 

If P S '^'^(/)j P is a final class of a stochastic matrix Q G df{u), for some 
eigenvector u of /. Subdifferentiating ( ^0|) aX x = u, we get, thanks to the technical 
assumption on Ai, ri and Pi, 

df,{u)=co{Pt \aeA} , 

see JRW98 , Th. 10.31]. Thus, we can write Qi as a convex combination, Qi 



J2a£A '^i^i'y where the af are such that af > 0, J2a£A- ~ 1' ^'^^ finitely 
many af are zero. Consider now the measure — X^aeA ^i^a (the associated 
randomized Markov stationary strategy t°° consists in playing the action a with 
probability af, when in state i, here 6a denotes the Dirac probability measure at 
a). Averaging the equalities X + Ui = rf + P°-u which hold for all a ^ Ai, we get 

(45) A + u = + P^u . 



Since P is a final class of the matrix P"^, restricting (45) to P and left multiplying 
it by TOp, we get A + m'^up ~ m'^rp + rUpPppUp, and since rupPpp = rup, we 
get A = mprp. 

Conversely, let us assume that P is a final class of a matrix P^ for some Mar- 
kovian stationary strategy r°°, and that mprp ~ A. For all a E Ai, wc set 

wf = X + u, - rf -Ptu>0 , 
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and define the vector w = {J^wfdTi{a))i^p. We have 
(46) w = X + Up — — PppUp . 

Left multipiying ( ^6| ) by mj, and using again the fact that rripPpp ~ rrip, together 
witli TTiprp ~ A, we get nipW ~ 0. Since all the entries of rrip are positive, and 
those of w are nonnegative, we must have w — 0. Hence fi(u) = rj + P[u, for 
all i ^ F, which implies that P[ G dfi{u), for all i E F. Taking any element 
Q e df{u) such that Qi = P[ for i E F, wc obtain that F is a final class of Q, 
hence F E'^'^if). □ 
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